Writing scenarios

Scenarios live as markdown under .scout/specs/**/*.scout.md — one file per feature/component, versioned and reviewed like the rest of your tests. The markdown is the source of truth and a pure input: a run never writes back to it. Status and last-run derive from .scout/runs/ instead, so the spec diff only ever reflects an intent change, never run noise.

---
feature: Paywall          # optional; defaults to the filename
profile: anon             # default auth profile for scenarios below
tags: [monetization]      # optional
viewports: [mobile]       # optional; screen sizes each scenario runs in
---

## Free user hits paywall on ep 3
Open ep 3 of series X without login; paywall appears with a signup CTA.

## Subscriber bypasses paywall
profile: qa               # per-scenario override (also: notes, tags, viewports)
viewports: [mobile, desktop]

Logged-in subscriber opens ep 3; plays with no paywall.

You state what a user can do and what must be true — never how to click it. That is the whole point: the sentence is the intent you hold the product to, and the agent discovers the real elements at run time so the wording survives a redesign.

Rules

• Each ## heading is one scenario. Its logical slug is <file>/<scenario> (e.g. paywall/free-user-hits-paywall-on-ep-3) and must be unique across the suite.
• Frontmatter (YAML, optional): feature (defaults to the filename), profile (default auth profile for the file), tags, viewports.
• Per-scenario overrides: profile:, notes:, tags:, viewports:, the permission keys (grantPermissions:, denyPermissions:, geolocation:), cookies:, storage:, and device: placed immediately under a heading (before the prose) override the file-level defaults.
• Body = flow + expected behavior, in plain language. Describe what the user does and what must (or must not) be true. No CSS selectors, no Playwright code — the agent discovers the real elements at run time and records them.
• A .scout.md whose every ## lives inside a fenced ``` block parses as zero scenarios (that’s how example.scout.md documents the format without polluting the suite).
• Duplicate headings in a file, or a scenario with no body text, are hard errors.

Viewports (screen sizes)

Each scenario runs in one or more named viewports. List more than one and the scenario fans out into independent verification units — each viewport gets its own recorded script (<slug>@<viewport>.json), its own verdict, and its own demo video. That is how you cover responsive behavior: the mobile hamburger and the desktop nav are different flows, verified separately.

Three viewports are built in, usable with no config:

• mobile — iPhone 13 emulation (touch + mobile UA), pinned to 390×844.
• desktop — 1280×800, no touch.
• tablet — iPad Mini (768×1024, touch).

A per-scenario viewports: list replaces the file-level one (it does not merge like tags) — each scenario explicitly chooses its sizes. Declare none anywhere and scout uses the config’s defaultViewport (mobile).

---
feature: Navigation
viewports: [mobile, desktop]   # file default: every scenario runs in both
---

## Primary nav reaches Pricing
Open the home page and reach the "Pricing" link from the main navigation.

## Hamburger opens the drawer
viewports: [mobile]            # only meaningful on small screens

Open the home page; tapping the menu button reveals the navigation drawer.

Names are limited to [a-z0-9-] (they become script-file tokens); a name not in the registry fails the run with a clear error. Override or add viewports in scout.config.json, and force one ad-hoc with scout go --viewport <name> (that run never persists a script). The demo video follows the viewport — a desktop scenario records a landscape clip.

Browser permissions

Flows that trigger a native permission prompt (geolocation, notifications, camera, microphone) can’t be driven by the agent — the prompt lives outside the page. Declare a policy and scout sets it at browser launch, for both the AI run and replay. Three states per permission: not declared → native behavior (untouched); grantPermissions → granted; denyPermissions → blocked, so no prompt appears.

Set it in the frontmatter (file default) and/or per scenario (overrides merge per-axis, and grant wins over deny):

---
feature: Store Locator
denyPermissions: [geolocation]      # file default: never prompt for location
---

## Search falls back to manual when location is blocked
Open the store locator, search for "Merate", confirm results appear.

## Nearby stores with a fixed location
grantPermissions: geolocation       # this scenario grants instead
geolocation: 45.69, 9.43            # required when geolocation is granted (lat, lng)

Open the store locator; the nearest store to the coordinates is shown.

Allowed: geolocation, notifications, camera, microphone, clipboard-read, clipboard-write, midi — an unknown name is a hard error. Granting geolocation requires coordinates (and coordinates imply granting it). denyPermissions matters mostly in headed runs (headless denies silently); grantPermissions and a granted geolocation change behavior in CI too.

Cookies

When a flow’s precondition is a cookie — forcing a server-side experiment variant, a feature flag, a consent state — declare it and scout seeds it into the browser before the first navigation, for both the AI run and replay. It’s a context-creation parameter (like storageState), never a recorded step, so replay stays deterministic and the agent never needs a “set cookie” tool.

Two forms. In the frontmatter (file default) or a profile (shared base in scout.config.json), cookies: is a list of objects — the place for attributes:

---
feature: Checkout
cookies:
  - name: hn_checkout_variant
    value: A                 # file default: variant A
  - name: consent
    value: "yes"
    httpOnly: true
    sameSite: Lax            # Strict | Lax | None
---

## Default variant
Open checkout; the file-level variant A is in effect.

## Force variant C
cookies: hn_checkout_variant=C   # per-scenario override: inline name=value

Open checkout; variant C is forced.

• Merge by name: a profile’s cookies are the base; the file frontmatter and then the per-scenario override win, keyed by cookie name. The per-## override is the terse inline name=value[, n2=v2] form (no attributes — those belong in the frontmatter/profile).
• name and value are required; domain, path, expires (unix seconds), httpOnly, secure, sameSite are optional. domain/path default to the host of baseUrl and /.
• Secrets: a value may use the $ENV:VAR placeholder (e.g. cookies: session=$ENV:SESSION_TOKEN) — resolved at launch, so the secret never lands in the committed spec or the agent’s context. Only value is resolved.
• Fail-fast: an unknown field, a bad sameSite, or a missing env var is a hard error — a silently skipped cookie precondition would produce a misleading verdict.

Local & session storage

Some features are gated not by a cookie or the server, but by the browser’s own web storage — an open-count threshold, a “you’ve seen this” flag, a dismissed prompt. The agent can’t set storage (it has no page.evaluate tool), so those scenarios used to stall at partial. Declare a storage: seed and scout applies it before the app loads, for both the AI run and replay. Like cookies/storageState, it’s a context-creation parameter, never a recorded step — replay re-resolves it fresh from the spec, so it stays deterministic and the agent never needs a “set storage” tool.

It seeds both localStorage and sessionStorage (the latter is something storageState can’t carry) via an init-script that runs before any page script, and it can remove keys to guarantee a clean precondition.

Two forms. In the frontmatter (file default) or a profile (shared base in scout.config.json), storage: is an object with local, session and/or remove:

---
feature: PWA install prompt
storage:
  local:
    hn_app_open_count: "2"      # file default: two prior opens
  remove:
    - hn_pwa_prompt_dismissed   # start from a non-dismissed state
---

## Below the threshold — no prompt yet
Open the app; the install prompt does not appear.

## At the threshold — prompt appears
storage: local.hn_app_open_count=3, remove=hn_other_flag

Open the app; the install prompt appears.

• Inline override form: the per-## override is a single line of comma-separated tokens — local.<key>=<value>, session.<key>=<value>, or remove=<key>. The namespace is spelled inline because a heading override line can’t carry a nested YAML object; nested objects belong in the frontmatter/profile.
• Merge: a profile’s storage is the base; the file frontmatter and then the per-scenario override win, per key per namespace. remove lists concatenate and dedupe across all levels.
• remove clears both namespaces: a removed key is dropped from localStorage and sessionStorage, so a stale value never leaks into the run. Removals apply before the seed, so a declared value always wins over a removal of the same key.
• Secrets: a value may use the $ENV:VAR placeholder — resolved at launch, so the secret never lands in the committed spec or the agent’s context.
• Fail-fast: an unknown field (only local/session/remove are allowed), a non-string value, or a malformed inline token is a hard error — a silently skipped storage precondition would produce a misleading verdict.

Device / user-agent emulation

Some UI is gated on device or user-agent detection — an “Add to Home Screen” sheet that only renders under an iOS-Safari UA, a layout branch that keys off touch support. By default scout launches desktop Chromium with its own UA, so those flows can never pass. Declare a device: and scout emulates it at browser launch, for both the AI run and replay. Like cookies/storage/storageState, it’s a context-creation parameter, never a recorded step — replay re-reads the frontmatter, so it stays deterministic.

device names a Playwright device descriptor (e.g. iPhone 14, Pixel 7). Individual fields — userAgent, viewport ({ width, height }), deviceScaleFactor, isMobile, hasTouch — compose on top of (or without) the named device; an explicit field always wins over the device’s value.

Two forms. In the frontmatter (file default) or a profile (shared base in scout.config.json), device: is an object:

---
feature: Add to Home Screen
device:
  device: iPhone 14          # a Playwright device name (the base)
  userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 …)"  # optional override
---

## Sheet renders under iOS Safari
Open the app; the Add-to-Home-Screen sheet appears.

## Same flow on Android
device: Pixel 7              # per-scenario inline override: a device name

Open the app; the native install banner appears instead.

• Inline override form: the per-## override names a device only — device: <device name>. Individual overrides (userAgent, viewport, …) belong in the frontmatter/profile because a heading override line can’t carry a nested YAML object.
• Merge: a profile’s device is the base; the file frontmatter and then the per-scenario override win, per field (an inline device: Pixel 7 keeps a file-level userAgent). The viewport field is replaced wholesale (it’s a width/height pair).
• Composes over the viewport: the resolved device layers on top of the run’s viewport — its viewport/UA/isMobile/hasTouch/deviceScaleFactor win over the viewport’s. The named viewport still determines which sizes the scenario fans out in and the run identity (<slug>@<viewport>).
• Not secrets: these fields are plain values, so there is no $ENV:VAR resolution (unlike cookies/storage).
• Fail-fast: an unknown field, an unknown device name, or a bad shape is a hard error at parse — a silently ignored device would produce a misleading verdict.

Flows that open a new tab

When a click opens a new tab or popup (a booking tool, an OAuth window), just describe it — the agent follows into the new tab and scout records the switch as a deterministic step. Console, network, and text assertions always run on the active tab, so a check after the switch verifies the new tab, not the one you came from.

Show/hide toggles (opacity, class, attribute)

Some controls aren’t added and removed from the page — they’re revealed and hidden in place, with opacity, a class swap, or an attribute flip, while staying mounted in the DOM. A plain “is it visible?” text check can’t tell those apart (a browser treats an opacity:0 element that’s still laid out as visible). Just describe the visual state and scout records a structural assertion — a class token (opacity-0 vs opacity-100), an attribute (aria-expanded), or a computed style — that replay re-checks deterministically.

## Menu button toggles the drawer
Tap the menu button; the navigation drawer becomes visible. Tap it again;
the drawer is hidden (it stays in the page, just faded out).

The agent also reaches elements that have no accessible role or name — a gesture/tap layer, an overlay <div> identified only by data-testid — which a role-based click can’t target. You don’t declare selectors; the agent discovers the real element at run time and records a stable data-testid (or CSS fallback) locator.

Asserting console logs & API calls

Beyond what’s on screen, scout can verify the browser console and the network calls the page makes. Just describe the expectation in the prose — the agent observes the real console/network on the first verified run and records a tolerant, deterministic assertion that replay re-checks without an LLM.

## Checkout fires the order API cleanly
The user completes checkout. A POST to /api/checkout returns 2xx and the
response includes an orderId. No errors appear in the browser console.

What the agent records:

• Network — matched by method + URL pattern (glob with */**) + status class (2xx), optionally requiring stable substrings in the response body (field names like orderId). It deliberately avoids volatile values (ids, timestamps), so the assertion survives replay.
• Console — “no errors” covers console.error and uncaught exceptions. Known/expected noise can be ignored by substring (e.g. a third-party favicon 404).

You can also assert a specific log appeared — handy for debug output gated behind a flag — not just the absence of errors:

## Debug logging turns on with ?gat-debug=true
Open the page with ?gat-debug=true. A console log containing "DEBUG:[GAT]" appears.

This records a positive console assertion that replay re-checks: it requires a single message containing your substring(s), so match on a stable prefix (DEBUG:[GAT]), never a volatile value — that keeps it tolerant of unrelated console noise.

Keep these expectations about shape, not exact values — “a POST to /api/checkout returned 2xx with an orderId”, not “orderId was ord_42”. Pinning a volatile value is the main way a network assertion turns flaky.

Simulating a transient network failure

To verify resilience — “a video segment fails to load and playback keeps going”, “the feed API drops once and the app retries” — describe the failure in the prose. The agent arms a network fault right before the moment that issues the request, lets the app recover, then asserts the recovered state:

## Playback survives a dropped segment
serviceWorkers: block

Open episode 1 and let it play. Make the next video segment (.ts or .m4s)
fail once with a network error. The video keeps playing past 0:15 and no
error message appears.

This records a failNetwork step. Its semantics:

• Armed when the step runs, never at launch — it only affects requests made after that point, in every tab of the context (a popup or switchTab target included).
• Mode — abort (the request fails as a network error, net::ERR_FAILED) or status (fulfilled with an HTTP error, default 503).
• Scope, exactly one — times: N fails the next N matching requests and then stops interfering (default 1, a single transient blip); durationMs fails every matching request for that window (an outage). The interception is removed when the budget or window is used up, and always when the scenario ends — later steps and later scenarios are unaffected.
• URL pattern — the same glob as network assertions (* within a segment, ** across, {a,b} alternatives), e.g. **/*.{ts,m4s} for HLS/DASH segments. Optional resourceTypes narrows it (xhr, fetch, media, …) — HLS players running on MSE load segments via xhr/fetch.
• Observable — failed requests show up in the network log marked as injected, and an assertNetwork with injected: true proves the fault really fired (aborted requests carry status 0). Plain network assertions ignore injected requests, so they can never pass on a response Scout fabricated. assertNetwork waits for a match (optional timeout, default 10 000 ms), so the proof holds on replay even when the request fires a moment after the step before it; a fault that never fires still fails, saying how long it waited.
• Never a silent no-op — result.json reports how many requests each fault failed (networkFaults). A fault that matched nothing fails the run at that step: a resilience check that never failed anything would be a false pass.

Replay is deterministic: the recorded fault carries its explicit scope (times: 1 when you gave none), so the same requests are failed in the same order, with no AI involved. A recorded script looks like this:

[
  { "kind": "navigate", "url": "/series/example/episode/1" },
  { "kind": "failNetwork", "urlGlob": "**/*.{ts,m4s}", "mode": "abort", "times": 1, "resourceTypes": ["xhr", "fetch"] },
  { "kind": "waitForText", "text": "0:15", "timeout": 30000 },
  { "kind": "assertNetwork", "urlGlob": "**/*.{ts,m4s}", "injected": true },
  { "kind": "assertNotVisible", "text": "Erro ao carregar o vídeo" }
]

Going offline

To test how the app handles losing the connection, a scenario can take the whole browser offline and bring it back:

## Offline notice and recovery

Open episode 1 and let it play. Go offline: a "check your connection"
notice appears. Go back online: the notice disappears and playback resumes.

This records setOffline steps ({ "kind": "setOffline", "offline": true }, then false). Offline applies to every tab: new requests fail, navigator.onLine flips and the page receives the offline and online events. Replay is deterministic. Use failNetwork when only some requests should fail.

Service workers

Set serviceWorkers: block (file frontmatter or per scenario) on scenarios that use a network fault when the app registers a service worker. Requests a worker answers from its own cache never reach the network, so no fault can touch them. Blocking is opt-in: scenarios without the key keep the browser’s default, so existing scenarios and PWA flows are unaffected. When a fault runs on a worker-controlled page, the report carries a warning naming this setting.

Authoring options

• By hand — write the markdown directly.
• CLI — scout create <name> -f <feature> -c <text> [-p profile] [-n notes]. A convenience for humans without an agent.
• AI agent — your coding agent writes a richer spec straight from repo context. See AI agents & MCP.

Migrating from a legacy scenarios.json

Older scouts kept a single .scout/scenarios.json. Convert it once:

scout migrate   # → one .scout/specs/<slug>.scout.md per scenario, relocates cached scripts, backs up the JSON

It’s idempotent and preserves cached scripts (so replay still works without re-recording). Re-run scout go once afterward to repopulate run status, review the generated feature: frontmatter, and delete .scout/scenarios.json.bak when happy.