Continuous video evidence: beyond replay-only analytics

Session replay made product behavior visible. The next useful layer makes recorded experience data queryable: not a single generated summary, but a repeatable evidence contract your backend can run across every recording it is authorized to process.

That distinction matters. “Add AI to replay” can mean a chat box over one session, a prose recap, or an automatic highlight reel. Those interfaces can be useful, but they do not automatically give an engineering team stable definitions, typed results, cross-run queries, or source-and-version provenance. A continuous video evidence system needs all four.

What continuous video evidence means

Continuous does not mean a hidden live feed. It means the analysis definition persists while recordings flow through your product over time. You define what counts as evidence once, version changes intentionally, run that definition against each new authorized source, and retain results in a form your application can query.

For example, a product team might define an onboarding-friction scanner around observable signals: a user repeats the same setup action, receives a visible error, or leaves the step without advancing. The same scanner can process this week’s recordings and next week’s recordings. Every result remains tied to the scanner version, run, source metadata, and exact evidence range that produced it.

The evidence loop: define, run, observe, act

The platform separates four concerns that are often collapsed into one opaque AI task.

1. Define: create a saved scanner with an evidence question, scanner type, output constraints, and bounded metadata.
2. Upload: place one authorized video or audio recording in the account-owned private upload namespace.
3. Run: apply the current version to that ready upload. The run snapshots the version so future edits do not rewrite history.
4. Observe: retrieve typed answers with explicit semantics, relevance, media kind, evidence kind, source metadata, and grounded timestamps.
5. Act: let your application search, aggregate, alert, or open a review item using the JSON.

POST /v1/scanners
POST /v1/scanners/{scanner_id}/runs
GET  /v1/runs/{run_id}
GET  /v1/runs/{run_id}/observations

# separate optional one-off media workflow
GET  /v1/analyses/{analysis_id}/moments
POST /v1/renders

This separation gives each resource one job. A scanner expresses a saved question. A run records execution. An observation records a typed, timestamped answer. Scanner completion does not create media; video find_moments observations include Moment IDs that a customer can explicitly render.

Why a saved scanner is more than a prompt

A copied prompt is mutable text. A scanner is a versioned API resource with an output contract. Updating its instructions creates a new version; prior runs retain the definition they used. That makes before-and-after comparisons possible and prevents an old result from appearing as though it came from today’s policy.

The scanner type also communicates what the result means:

• monitor returns boolean / match_found, including an explicit false only after a completed full-source review supports no match.
• classifier returns string / classification_label from configured labels, or other.
• scorer returns number / relevance_score from 0–1 under the requested evidence criterion.
• summarizer returns string / moment_summary as a whole-source answer.
• extractor returns object / extracted_fields grounded in source evidence.
• find_moments returns object / timestamped_moment once per supported timestamped evidence window.

Those outputs should not be flattened into a universal “AI score.” Each observation carries value_type and semantics so a client can interpret it correctly. The separate relevance_score ranks how strongly the result supports the scanner instructions; it is not calibrated confidence.

Evidence should survive the summary

A summary can tell a team that onboarding looked difficult. Evidence tells the team where the user repeated an action, what error appeared, what the user said, and which source recording contains the event. That difference determines whether someone can verify a finding or build a reliable downstream workflow.

Timestamped observations preserve start_ms, end_ms, and duration. Video find_moments returns editorially selected moments; other scanner types answer the focused question in their declared value type. Audio-generated speech excerpts are not certified verbatim transcripts.

A completed unanswerable run has zero observations because available source evidence could not support an answer. That is distinct from a negative answer: a supported monitor no-match produces exactly one boolean false observation covering the full recording after completed review. Never rewrite an unanswerable zero into false.

How this differs from replay-only products

Replay products and a video evidence API solve overlapping but different problems. A mature replay product may own browser capture, DOM reconstruction, network and console telemetry, user/session lookup, funnels, heatmaps, and an analyst-facing playback interface. ChatClipThat does not claim those capabilities.

If your main need is automatic browser capture, DOM-level debugging, network waterfalls, heatmaps, or funnel analytics, use a session replay product. If you need a reusable server-side evidence contract across video or audio your systems already control, this API adds a different layer. The two can complement each other.

A practical continuous architecture

A production integration usually has six small components rather than one large “AI” service:

1. Source adapter: enumerates authorized recordings and creates completed private uploads.
2. Submission queue: assigns a stable idempotency key per upload, scanner, and intended version.
3. Bounded worker: stays below the concurrency and monthly allowances shown in Console → Usage.
4. Terminal-state reconciler: polls every 5–10 seconds with jitter, or consumes signed webhooks when delivery is enabled, while still reconciling missed events.
5. Observation store: deduplicates by observation ID and preserves typed value, semantics, evidence, and provenance.
6. Decision layer: powers search, reporting, review, tickets, or alerts from JSON.

recording discovered by your system
        ↓
queue row + deterministic idempotency key
        ↓
scanner run against ready private upload
        ↓
completed / failed terminal state
        ↓
typed observations + exact evidence
        ↓
search · aggregate · review · alert

The client should pause on validation, authorization, source-host, quota, or insufficient-credit errors until state changes. It should retry transient dispatch or network failures with bounded exponential backoff. Secrets, raw signed URLs, and authorization headers should never enter logs or queue metadata.

Optional rendering changes the economics of the workflow

Automatically rendering every result is expensive operationally even when the render has no additional API credit cost: it creates files nobody may watch, forces layout decisions too early, and turns search infrastructure into a media-production queue.

ChatClipThat scanner runs return structured observation JSON and never render automatically. A review dashboard can use typed values, timestamps, titles, summaries, and metadata directly. When finished video is required, take the moment_id from a completed video find_moments observation and call POST /v1/renders. An observation ID is not a Moment ID.

This boundary is especially useful for continuous analysis. Thousands of sources may produce hundreds of relevant observations, while only a handful need a shareable clip. Analysis remains the evidence layer; rendering remains a deliberate publishing step.

Credit and quota behavior stays visible

Scanner runs and one-off analyses use the shared ChatClipThat wallet at one credit per minute analyzed, rounded up to the next minute. A selected video find_moments window is billed by its duration; a run without a window is billed by the measured full-upload duration. Status polling, observation and Moment JSON, render history, presets, and optional rendering do not consume additional API credits. API credentials currently require a paid ChatClipThat account, with included proof-run boundaries documented on the site.

There is no universal throughput promise hidden in this article. Monthly source-minute, run, concurrency, key, trusted-host, and retention limits are account-specific and shown in Console → Usage. Your queue should read and respect those boundaries.

Design the scanner around what a reviewer can verify

The strongest scanner instructions describe something the selected media can support. For video, “find where the user repeats the same action or receives a visible error” defines a reviewable event. For audio, ask for spoken objections or needs; never ask audio to establish an interface state it cannot show. All six scanner methods support both media kinds.

Use narrow scanners when downstream behavior differs. An alerting monitor, an onboarding-friction moment finder, an objection extractor, and a conversation summarizer may process the same recording, but they should not pretend to answer the same question. Version each definition when the standard changes.

Privacy and authorization are application responsibilities

Only submit recordings you are authorized to process. Keep private upload access server-side and avoid placing personal data or secrets in metadata. The visual worker reads only the account-owned, generation-pinned private object selected by the API; your application still owns consent, disclosure, access policy, retention, and deletion obligations.

For interviews, support calls, and user research, an AI evidence pipeline does not replace consent management. For product decisions, generated observations should support review rather than become an unexplained high-impact decision engine.

Where to start

Pick one modality-appropriate question and ten representative recordings. Create the scanner, preserve media and evidence kinds, inspect explicit negative answers separately from unanswerable zero-result runs, and revise the definition as a new version. Once stable, connect it to your queue and review workflow.

The result is not “AI watched a video.” It is a durable application primitive: the same question, applied repeatedly, producing typed evidence that can be verified, queried, and acted on without forcing every result into a clip.