Onboarding friction rarely looks like one universal event. It appears as a sequence: hesitation, a repeated action, an error or confused comment, and either recovery or abandonment. A scanner should preserve that sequence.
Start with one falsifiable question
Broad prompts make triage harder. Separate “could not connect an integration,” “could not find the next step,” and “abandoned after pricing” into different scanners. Each scanner can evolve independently while old runs retain their original version.
{
"name": "Integration connection blocked",
"type": "find_moments",
"instructions": "Find where the user attempts to connect an integration, receives an error or no visible progress, and retries or leaves without advancing. Preserve the attempted action through the outcome.",
"profile": "conversation",
"max_observations": 10,
"min_seconds": 10,
"max_seconds": 75
}Carry cohort context as metadata
Add scalar fields such as plan, acquisition channel, experiment, app version, or workspace. They are returned with observations and can be used by your system to compare cohorts without placing private identifiers inside scanner instructions.
Review the evidence, not only the score
Use relevance to order a review queue. For product decisions, inspect the grounded visual evidence range and typed value. A relevance score tells you how closely a finding matches the brief; it does not measure the probability that your product caused the behavior.
Close the loop
Store observation IDs beside the issue or research note they inform. If a design changes, create runs with the same scanner and new-version metadata. Comparing cohorts is currently an integration responsibility; ChatClipThat exposes the structured records needed to build it.