Tutorial Video Workbench

Architecture and rationale live in tools/tutorial_videos/README.md — read
it first. This skill is the operational runbook.

Build videos

make tutorial_video TUTORIAL_LOCALE=de                       # one locale (~5-8 min)
make tutorial_video TUTORIAL_LOCALE=de TUTORIAL_DEVICE=mobile # phone-shaped window, `_mobile`-suffixed output
make tutorial_videos_all                                     # all TUTORIAL_LOCALES

TUTORIAL_DEVICE (desktop|mobile, default desktop) picks the capture
window size — see the README's "Desktop vs. mobile" section for the
breakpoint rationale and the tutorialDeviceIsMobile() test hook.

Preconditions (fail fast if missing):

• .env at repo root with GEMINI_API_KEY, MELIOUS_API_KEY,
  MELIOUS_BASE_URL. If absent, the keys can be extracted from the dev
  app's ai_config.sqlite provider rows (never print or commit them).
• Sibling ../OpenMontage checkout at the commit pinned in
  tools/tutorial_videos/config/openmontage.pin, bootstrapped via
  make -C ../OpenMontage setup.
• Xvfb, ffmpeg, pactl, paplay, x11-utils installed; Flutter via
  fvm.

Output lands in build/tutorial_videos/<scenario>_<locale>.mp4. Verify with
frame extraction (ffmpeg -ss <t> -frames:v 1) and check: cursor visible and
gliding, HUD clock top-center, transcript/proposals on screen, narration
audible at step starts (volumedetect), duration ≈ warped timeline total.

Publish to R2

make tutorial_video_publish TUTORIAL_SCENARIO=create_task_from_audio TUTORIAL_LOCALE=de
make tutorial_video_publish TUTORIAL_SCENARIO=create_task_from_audio TUTORIAL_LOCALE=de TUTORIAL_DEVICE=mobile

Uploads build/tutorial_videos/<scenario>_<locale>.mp4 (or, with
TUTORIAL_DEVICE=mobile, <scenario>_<locale>_mobile.mp4) to Cloudflare R2
at the matching tutorial-videos/... key and prints its public r2.dev URL —
this is the URL the docs-site TutorialVideo component builds from
TUTORIAL_VIDEO_BASE_URL/tutorialVideoBaseUrl. TUTORIAL_DEVICE must
match whatever --device the video was built with, or publish will look for
a file that was never produced. Full one-time Cloudflare setup (API token,
enabling the bucket's public r2.dev URL) and the .env variable list are in
the README's "Publishing to Cloudflare R2" section — read it before the
first publish.

boto3 (needed only by publish, not by build/tts/validate) is not installed
into the system Python on this host (PEP 668). Either run publish through
tools/tutorial_videos/.venv (.venv/bin/python3 -m tutorial_videos publish --scenario ... --locale ..., created via python3 -m venv .venv && .venv/bin/pip install boto3 pyyaml), or install boto3 into whatever
Python make tutorial_video_publish resolves to.

The pipeline in one paragraph

python3 -m tutorial_videos build (run from tools/tutorial_videos/) does:
Gemini-TTS pre-pass (cached by content hash → durations manifest) → Xvfb +
virtual-mic null sink + ffmpeg x11grab capture → fvm flutter drive runs
integration_test/tutorial/<scenario>_tutorial_test.dart against the real
app (penguin demo world + Melious/Voxtral/Qwen agent config seeded, UI
language via MANUAL_LANGUAGE settings row) → the test emits
timeline.json (step boundaries, wait spans, zero epoch) → timewarp.py
plans fast-forward segments for the waits (narration never overlaps) →
compose.py warps the capture and drives the pinned OpenMontage headlessly
(narration placement + mux) → ffprobe validation.

Debugging a failing run

1. The run log tail shows the failing step and a Diagnostics: line
   (current route, detail stack, key widget presence).
2. Look at the failure screenshot first:
   build/tutorial_videos/failure_<context>.png is captured at the moment of
   every timeout — it almost always explains the failure instantly.
3. Reproduce faster without video: run the same test via flutter drive
   directly (see the doc comment in
   integration_test/tutorial/tutorial_harness.dart for the env contract);
   Xvfb + sink setup must match session.py.
4. Common causes, in order of historical frequency: a finder matching an
   offstage duplicate (scope to the visible page + .hitTestable()), a
   scroll helper given a non-vertical scrollable, a modal/dialog blocking
   the UI (onboarding, discard-recording), the agent's
   automaticUpdatesEnabled flag not sticking (set-and-verify), cloud
   latency beyond a pumpUntil timeout.

Adding a locale

1. Add the locale's text to
   tools/tutorial_videos/config/scenarios/<scenario>.yaml: title,
   dictionary, every step's narration, the dictation step's
   dictation_text. Write informal register (du/tu) per repo l10n rules;
   keep the penguin vocabulary.
2. Add per-stream style lines for the locale in config/voices.yaml
   (style prompts in the target language — Voxtral/Gemini behave better).
3. python3 -m tutorial_videos validate --scenario <s> --locale <l> must
   pass, then build. The app UI localizes itself via LOTTI_MANUAL_LOCALE
   (same mechanism as manual screenshots; for a wholly new app locale, run
   the add-flutter-docusaurus-locale skill first).
4. Localized finder strings in the test (localized(en:, de:) /
   manualScreenshotText) need the new locale only if its UI labels are
   used as finders — prefer keys over text finders where possible.

Adding a scenario

1. New YAML in config/scenarios/<name>.yaml — schema documented in
   create_task_from_audio.yaml (unique step ids, exactly one
   dictation: true step, per-locale text, min_duration floors).
2. New integration_test/tutorial/<name>_tutorial_test.dart built on
   TutorialAppHarness + TutorialDriver. Copy the structure of
   create_task_from_audio_tutorial_test.dart:
   • one driver.step(id, action) per YAML step, ids matching exactly;
   • interactions via driver.tapLikeUser (cursor glide), waits via
     driver.pumpUntil* (records fast-forwardable wait spans), scrolling
     via driver.scrollIntoView with a vertical scrollable finder;
   • anything that must be SEEN gets asserted via visible widgets, not just
     the DB;
   • wire the diagnostics and onTimeout hooks for failure screenshots.
3. Iterate with builds; expect the first runs to fail — read the failure
   screenshots, they carry the answer.

Non-negotiable gotchas

The full list with rationale is in the README's "Hard-won constraints" —
the ones that bite hardest: flutter drive not flutter test (no window
otherwise); GDK_BACKEND=x11 + empty WAYLAND_DISPLAY (or the app opens on
the real desktop); Voxtral prompts in the audio's language (else it
translates); automaticUpdatesEnabled set-and-verify (initial wake clobbers
early writes); 16ms pump cadence during holds (else animations render
choppy); timeline starts after startup settles; assert transcript keywords,
not exact strings.

Adapting / extending

• App Store preview: config/scenarios/app_store_preview.yaml is the
  narration of make store_preview_ios, not a tutorial — never
  make tutorial_video it. Its step ids must stay the _beatIds of
  integration_test/store_preview_test.dart (tests/test_app_preview.py
  guards it), and all four lines together must leave the beats under 30 s
  (python3 -m tutorial_videos.app_preview pacing fails first if not).
  See the README's "The App Store preview reuses the narration".
• Compositor changes: all OpenMontage API usage is isolated in
  tutorial_videos/om_compose_driver.py; bump the pin in
  config/openmontage.pin deliberately and re-verify determinism
  (compose twice, compare ffmpeg -f framemd5 of the streams — never file
  hashes).
• TTS vendor: implement the TtsEngine protocol (tts/base.py) next to
  tts/gemini.py; voices/styles per locale in config/voices.yaml.
• Speed/pacing knobs: timewarp.py (MAX_SPEED, lead-in, narration
  gap); per-step floors in the scenario YAML.
• Character overlay (future): render transparent PNG frames via the
  character film-strip harness and composite as an extra layer keyed to
  timeline.json step ids.