"Deep, infrequent, human-attended review of a WHOLE project's architectural and code quality against docs/code-quality.md — pins one tree, takes a whole-repo metrics snapshot (a report, never a gate), refreshes the stale instruments in delta mode (/hotspot-survey --refresh, /review --since) and consumes the census codebook, then runs a heuristic-driven architecture pass: one seat per area judging all fourteen heuristics and both stances, cross-area seats for SPOT / layering / orthogonal dimensions / stateless interactions, a skeptic per seat blinded to proposals, a synthesis joined by finding key with every prior instrument's open findings, and a critic of that synthesis; ends in deliberation with the human, a program doc and /prd per stream, and files the completion-driven trigger for the next run. ALWAYS use this skill for: '/review-all', 'review the whole project against the quality heuristics', 'whole-repo architecture and code-quality review', 'what should we refactor across the whole codebase', 'the quarterly factory self-review', or when the human gate task 'Run /review-all on <project>' dispatches. This is a serious commitment — the first dark-factory run measured 22.9M subagent tokens (8.6M of them in the Phase 2 instrument refreshes), 8 h of machine time for phases 1–4 and one full deliberation sitting plus the /prd work — so state the cost, Phase 2 included, and get a go before phase 2 (`--full` or `--yes` skips only the wait, never the posting). NOT for: does-it-run correctness of landed work (/review), bug-history hotspots alone (/hotspot-survey), confusion sightings (/census), one module (/study), one bug (/deb), authoring the remediation PRDs (/prd — fed by this skill's program doc), or anything unattended."
Resources
2Install
npx skillscat add leodearden/dark-factory/review-all Install via the SkillsCat registry.
/review-all — whole-project quality review
Answers one question about a whole project: where does the next agent change cost the most, under docs/code-quality.md, and what would remove that cost? Every other quality instrument answers a narrower question from narrower evidence; this one reads them all first (docs/quality-findings-contract.md §9), runs only what is stale, adds the one pass nothing else does — judging every area against the fourteen heuristics and both stances — and ends in human deliberation. It supersedes the "quarterly factory self-review" priced in the agent-capacity study. Cadence is task completion, not the calendar (contract §11).
Phase 0 (you) pin as_of_sha; areas; instrument ages + open findings; cost gate
Phase 1 (you, scripts) whole-repo metrics snapshot — report, never ranking
Phase 2 (sibling skills) /hotspot-survey --refresh, /review --since (census consumed, never run)
Phase 3 Workflow ~20+4 seats area seats → blinded skeptics → cross-area seats → skeptics
Phase 4 Workflow 2 seats synthesis → critic (one revision)
Phase 5 (you) digest, mint keys, report + findings JSON, file mechanical findings
Phase 6 (you + human) deliberation → program doc → /prd per stream → trigger chainParse invocation
/review-all → whole project, all phases, human present
/review-all --plan → phases 0–1 and the roster + cost only; no seats launched
/review-all --yes → accept the recommended roster at the cost gate and the recommended option at every deliberation item without waiting
/review-all --full → post the cost gate in full and proceed without waiting for a go; nothing else changes
/review-all --areas orchestrator,shared → restrict phase 3 to these briefing areas (cross lenses still run)
/review-all --fable → pre-authorise the synthesis and critic seats on fable
/review-all --no-file → report only: no tickets, no trigger chain
/review-all --launched-from <esc-id> → the review_due escalation that spawned this run (the normal launch path); omit or `none` for a hand launch
/review-all --resume <run_id> → resume the workflow from its journal (same args file); it re-runs every call after the first divergence in journal order, so a changed or unrun tail is re-run as a standalone workflow over the completed results instead (references/orchestration.md failure mode 16); in Phase 6 it resumes the program instead (§Lead handoff)A trailing path is the project root; otherwise git rev-parse --show-toplevel.
--plan and --yes are mutually exclusive (one stops at the gate, the other walks through it); given both, stop and say so. --yes changes no roster and no item shape: the cost gate is still posted in full and the run proceeds at once; in Phase 6 every item is still put in references/deliberation.md's fixed shape with its recommendation, and the recommended option is recorded as the ruling with ruled_by: lead (--yes) in the program doc §Resolved decisions and in its decisions_and_rationale memory. An item whose analysis does not converge (no recommendation) still stops for the human; --yes never auto-accepts an "accept as standing" disposition and never commits the trigger-chain gate (commit_planning stays the human's act).
--full skips the cost gate's wait and nothing else: the gate is still posted in full (roster, tokens, wall clock, human hours) and the run proceeds at once; the weekly-headroom rule still holds. It excludes --plan (given both, stop and say so) and does not imply --yes: every Phase 6 item still waits for the human. The skill is expensive by design, and an operator who asks for --full has already priced it; holding the session for a go they gave up front spends their attention for nothing.
Step 0 — overlay and prerequisites
ROOT="$(git rev-parse --show-toplevel)"
SKILL_MD="$(readlink -f ~/.claude/skills/review-all/SKILL.md)"
[ -f "$SKILL_MD" ] || { echo "review-all is not installed at ~/.claude/skills/review-all: set SKILL_ROOT to the skill's checkout by hand"; exit 1; }
SKILL_ROOT="$(git -C "$(dirname "$SKILL_MD")" rev-parse --show-toplevel)"
[ -f "$SKILL_ROOT/skills/review-all/SKILL.md" ] || { echo "SKILL_ROOT=$SKILL_ROOT holds no skills/review-all: set SKILL_ROOT to the skill's checkout by hand"; exit 1; }
ls "$ROOT/.claude/skills/review-all/project.md" 2>/dev/null && echo "overlay present" || echo "no overlay — generic mode"
ls "$ROOT/review/briefing.yaml" "$ROOT/docs/code-quality.md" "$ROOT/docs/quality-findings-contract.md"Read the overlay if present (references/project-overlay.md); it supplies identity, path→area rules, the per-language metrics instruments, slice seeds, report homes and hand-off conventions. Read docs/code-quality.md in full yourself before anything else: its Definition is the filter every seat applies, and its Do not steer by list binds you as the lead.
No briefing → stop. Area keys are the briefing's subprojects keys plus repo and there is no second list (contract §3); a run without them mints no joinable keys. Route to /review-briefing and return.
Cost gate
This is a human-attended skill; AFK autonomy does not apply. After Phase 1, post the roster (seat, model, effort, read budget, expected tokens) with the total token and wall-clock estimate and the human-hours Phase 6 implies, then wait for a go (--yes or --full: post it and proceed). A first run on a mature repo is a large commitment by design — say so plainly rather than trimming the roster to look cheap. --plan stops here.
The gate prices both halves of the run; the phase 3–4 roster alone understates it by more than a third:
- Phase 2, from each sibling skill's own gate:
/hotspot-surveyrefresh ≈ 3 + ⌈P/8⌉ + 2R + 1 agents and 0.3M + ⌈P/8⌉ × 75k + R × 195k + 140k tokens (its SKILL.md §Cost gate; P = prior findings re-verified, R = clusters reviewed), and/reviewhas no formula — quote its last measured run. A first refresh after a long gap re-reads every cluster and costs near a full hotspot run: the formula under-estimated the first dark-factory refresh by about 60 %. An instrument that has never run under the contract here is priced whole (§Phase 2). - Phases 3–4, from the roster in §Phases 3–4 and the per-seat figures in
references/orchestration.md§Model / effort routing.
Before posting the gate, check the account's remaining weekly usage headroom against the total estimate, Phase 2 included, and say at the gate whether the run fits. Measured on the first reify run (review-all-reify-20261010): the weekly limit fired mid-Phase 3 at 11.2M subagent tokens and 37 of 70 agents failed; the resume then re-ran ~1.2M tokens of completed calls (references/orchestration.md failure mode 16). A run that does not fit waits for the reset.
Calibration, the first dark-factory run (review-all-dark_factory-20261008, measured): Phase 2 8.6M tokens — hotspot refresh 5.66M over 89 min with 42 agents, /review full 2.97M over 133 min with 10 agents; phases 3–4 14.2M tokens over 50 seats; total 22.9M; wall clock 8:05 for phases 1–4; Phase 6 one sitting for 26 streams, then six /prd seats in wave 1. Quote these beside the formula figures until a second run replaces them.
Calibration across the three runs since (reify review-all-reify-20261010, solar_challenge_platform and solar_challenge …-20261013; the 2026-10-17 cross-run feedback synthesis), measured: phases 3–4 wall clock 4.4–7.2 h, not the 1.5–2.5 h a gate quoted — phase barriers and usage-cap waits set it, not seat count. Findings per 1M Phase 3–4 tokens ran 13–23 across all four runs, these three and the first (reify ≈ 23/M, net of 14.4M tokens interrupted and re-run); source size is not a budget (27–158 tokens per source line), so price the roster, not the line count. Seat figures: area ~225k, cross ~205k, skeptic 90–115k, synthesis ~320k, critic ~500k (solar_challenge 494k, 2.5× the figure in references/orchestration.md §Model / effort routing). Count tokens by one rule and state it in ## Method (references/report-format.md §Artefact 1) as extra.token_rule (which seats are summed, whether interrupted and re-run calls count), so runs compare. Human hours, which the gate must post: solar_challenge_platform ≈ 1.5 working days plus a day of rulings; solar_challenge 4 attended days, ~35 terminal sessions, 45 human rulings; reify ≈ 2.5 days plus a second sitting.
Phase 0 — pin, areas, instrument ages (inline)
- Pin one tree.
AS_OF=$(git -C "$ROOT" rev-parse main);run_id = review-all-<project_id>-<YYYYMMDD>[-<n>];since= the previousreview-allreport'sas_of_shaornone. Create a detached read-only worktree every seat will read — main moves under a live merge queue and a multi-hour run must read one tree. Pin it under the run's state home,TREE=~/.claude/review-all/<run_id>/tree: nothing sweeps that home, whereas a project-local.worktrees/copy was removed mid-run on solar_challenge and the.claudesymlink into it dangled:git -C "$ROOT" worktree add --detach "$TREE" "$AS_OF". Verify it resolves (git -C "$TREE" rev-parse HEADequals$AS_OF) before Phase 1 and re-verify it before each later phase that reads it; if it does not resolve, rungit -C "$ROOT" worktree prune(a removed directory stays registered and blocksworktree add) and recreate it from$AS_OF; where that fails, setTREE=<scratch>/review-all/<run_id>/tree, extractmkdir -p "$TREE" && git -C "$ROOT" archive "$AS_OF" | tar -x -C "$TREE", and record the substitution in the report's §Dropped and degraded (an archive has no HEAD; its check is that$TREEexists). Every later use of the tree is$TREE— the Phase 1--root, the Phase 2 siblings, the seats'root, the Phase 5 removal — and the record keeps it once, asextra.worktree.path. Never stash (CLAUDE.md §Working in the main checkout). Note the skill's own commit sha, in the skill's checkout rather than the target's ($SKILL_ROOT, Step 0):SKILL_SHA=$(git -C "$SKILL_ROOT" log -1 --format=%H HEAD -- skills/review-all), forextra.skill_sha(§Lead handoff); an emptySKILL_SHAis a stop, never a blank to record. - Areas. Parse
review/briefing.yamlsubprojectskeys; addrepo. Member lists inreview/briefing.yamlare a hard precondition: when the briefing names its members only in prose, stop and run/review-briefing; only on the human's explicit go-ahead does the run proceed on the overlay's inferred rule list below, recorded as a briefing defect — a stopgap, never the standing rule. One rule maps a path to an area (contract §3): the subproject whose member directory contains it; for paths under no member, the rule the overlay names. A path no rule covers is either a briefing defect (a workspace member or top-level package the briefing omits — report it for/review-briefing, exclude it from the run) or, when the overlay says so,repo; the report lists both. On that go-ahead the overlay carries the mapping as an ordered rule list — first match wins, a catch-all per area, sub-package rules where one package spans areas by file — and lists every mapping it inferred rather than read from the briefing as a briefing defect for/review-briefing; the snapshot records the rules in itsparamsso the report can cite them. The one rule is passed explicitly — written out, not left for each sibling to re-derive — on the invocation lines of the Phase 2 siblings (a differing rule gave/review, the hotspot survey and the run different keys at the same anchors on solar_challenge) and recorded in## Method's yaml block (references/report-format.md§Artefact 1) asextra.area_rules. Measured on the first reify run (review-all-reify-20261010): the briefing had purpose prose and no member lists, and two crates (reify-eval,reify-stdlib) were split across several areas by file. - Instrument ages and open findings. For each of the latest hotspot findings artefact,
/reviewreport, census report +docs/legibility/confusion-codebook.yaml, and previous/review-allrecord: noterun_id,as_of_sha,git rev-list --count <its sha>..$AS_OF -- <source globs>, and collect every finding withdispositionopenorfiled:(→ the prior open set) andaccepted:(→prior.standing), joined on contract §2canonical. A prior report with noas_of_sha(pre-contract) is stale by definition — no commits-since can be counted — and is recorded ininputs_consumedaslegacy:<path>. A legacy markdown-only report (a hotspot survey with no findings JSON) cannot be refreshed at all; convert it by hand — each diagnosis becomes prior findings with file-grain anchors,sub_area= the diagnosis label, keys minted byskills/hotspot-survey/scripts/findings_artefact.py key— so the area seats re-verify them like any prior, and list it aslegacy:<path>(first reify run: seven diagnoses became 20 priors; at the interim 7 still present, 4 gone, 8 unchecked). A defect in a sibling instrument found here (its report home gitignored against contract §5, a mis-named worktree) is filed as a task against that skill withsource="review-all", never fixed inside this run. A legacy<area>/<sub>area becomes the plain area in the key plussub_area, and the finding is assigned to the slice holding the sub-area's files (else the area's first slice) so it is re-verified, never left unchecked. Entries with no key and no anchor becomeleads. Staleness verdict per instrument: hotspot stale at ≥ 50 source commits since or when its human-gate task isdone; review stale on any source change (its--sincemode is cheap enough to always run). The census is never run here. - Memory.
search(query="architecture decisions, standing quality rulings, accepted gaps", project_id=<overlay>)→leadswithsource: memory. - Launch provenance. Once
plans/completion-driven-triggers-prd.mdlands, the normal way this skill starts is areview_dueescalation from the previous run's human gate (metadata.trigger_chain.skill == "review-all", the bare skill name), and its id reaches the run as--launched-from <esc-id>; record it in the report'sextra.launched_from— Phase 6 resolves it perskills/_shared/filing-the-trigger-chain.md§5 step 2. A hand launch recordsnone.
Phase 1 — metrics snapshot (inline, scripts only; no LLM judgment)
The instrument is scripts/quality_metrics_snapshot.py (contract: plans/quality-metrics-snapshot-prd.md §Contract), and its measures live in scripts/source_measures.py; the overlay names other languages' instruments. It is a report: no verdicts, ranking, averages or thresholds beyond heuristic 14's two marks, and the quality doc's Do not steer by list is what it must never be read as.
A project that is not a Python workspace or single package. The Python snapshot measures a uv workspace's members, or a single src-layout package (no [tool.uv.workspace]: [project].name with src/ and tests/), as scripts/source_measures.py::workspace_domain defines; on any other project the overlay's Metrics instruments slot names the instrument and it replaces step 2. For a Cargo workspace (with TypeScript, Python and shell beside it) that is scripts/rust_metrics_snapshot.py in this skill: python3 <skill>/scripts/rust_metrics_snapshot.py --root "$TREE" --run-id <run_id> --out <scratch>/review-all/<run_id>/ writes snapshot.json, import_graph.json and metrics_summary.txt there, after its PROJECT RULES block is replaced by the overlay's (it has no --diff yet). Its complexity figure is lizard cyclomatic complexity (CCN) per function, read as the pair (max per function, file total) — not the quality doc's cognitive complexity, which has no working Rust binary (rust-code-analysis-cli does not build) — so the slice brief's cognitive columns carry ccn_max/ccn_total labelled CCN, and the report's metrics section says so. Measured on the first reify run (review-all-reify-20261010): 2,901 files / 1.99M lines in 5m25s, 195 files ≥ 2,000 lines, where the Python default would have measured about 2 % of the code.
- Previous snapshot. The committed
plans/quality-metrics/*.jsonwhoseas_of_shaissince, else the newest whoseas_of_shais an ancestor ofAS_OF(git merge-base --is-ancestor). With none, omit--diff; the script then says there was no previous snapshot. The report names the snapshot it diffed. - Measure the pinned tree with the main checkout's script and venv, into scratch:
It refuses a dirty domain or a HEAD that moved during the run, exiting 2 with the cause named on stderr. Exit 2 stops Phase 1: report the named cause. There is no inline-probe fallback. If the snapshot'suv run --directory "$ROOT" python scripts/quality_metrics_snapshot.py \ --root "$TREE" --run-id <run_id> \ --out <scratch>/review-all/<run_id>/snapshot.json [--diff <previous>] \ > <scratch>/review-all/<run_id>/snapshot-diff.txtevidence.completeis false, the report listsevidence.unreadable: those files' measures are unknown, never zero. - The seats' inputs, under
<scratch>/review-all/<run_id>/:import_graph.json: the snapshot'simport_graphsection copied unchanged, e.g.python -c 'import json, sys; json.dump(json.load(open(sys.argv[1]))["import_graph"], open(sys.argv[2], "w"))' snapshot.json import_graph.json. Its nodes are module import names; a file record'smodulejoins them to paths.metrics_summary: the--summary <snapshot>output, followed by the sections ofsnapshot-diff.txt, trimmed to ≤ 4k chars by dropping whole entries from the end of a section (noted(+N more)), never re-sorted.
- Home. The script writes and never commits (the metrics PRD's decision 8). The snapshot is copied to
plans/quality-metrics/<run_id>.jsononly in Phase 5, immediately before that phase'sgit commit --only, so no uncommitted file sits in the machine-operated checkout.
Task 5414 is done (cdc15a6b56, 2026-10-05): its per-package pinning counts live in scripts/suite_census_pinning.py, which after the metrics PRD's S1 enumerates through scripts/source_measures.py::workspace_domain as the snapshot does — reuse those counts rather than re-deriving them.
Slices. Hand-author the slice list (never delegate it): one seat per area; an area over ~40k source lines is split along the import graph's package clusters (overlay seeds first) into slices of ≤ ~40k lines. Pack by source lines, never total lines: pack src clusters to the cap and attach a package's tests to the slice holding most of its src (as index-only entries plus tests_reaching_internals); a test suite that is itself its area's product (reify's tests/infra) counts as src; an area with one small file is read inside a neighbouring slice, its READ FULLY entries carrying that area, and the prompt tells the seat. Measured on the first reify run: packing by total lines gave 61 slices, 20 of them pure-test seats with nothing to read fully; by src lines, 30. Within a slice, read_fully is the top ~10k lines (what a seat actually reads; below) ranked by cognitive max, fan-in, reach-back imports, cycle-closing deferred imports (cycle_closing_imports), re-exports, patch targets and fix density — never by line count or the raw deferred-import count — with each entry's numbers taken from the snapshot's file records (the mapping is in references/orchestration.md §The slice brief), and index_only is the rest. tests_reaching_internals comes from tests records' private_patch_targets; size_alarms from records whose lines reach params.h14_alarm_lines; prior_open and leads from Phase 0, filtered to the area.
Rotation. The ranking instruments move slowly, so without a rotation the same files top every run's read lists and index-only files stay index-only — a blind spot the method itself creates. Read read_coverage from the previous /review-all record (the newest plans/review-all-<project_id>-*.json whose as_of_sha is an ancestor of AS_OF; none → no rotation): a file a seat listed under files_read_fully or partial last run has its percentile-rank sum multiplied by 0.5, unless fix-grepped commits have touched it since that record's as_of_sha, in which case it keeps its rank. The report's read-coverage table then says how many files were read fully for the first time (references/report-format.md §Artefact 1).
Read shortfall. Size read lists at what a seat reads, not at the plan. Measured on the first reify run: 30 seats were handed 298 src files / 427k lines (15k each) and reported 234k read — 55 % of the plan, 27 % of 880k src lines; 12 seats read under half their plan, typically 4–8k lines of two 13–15k-line files before stopping; of the 16 files with a per-function CCN max ≥ 60, 9 were on a read list and none was reported read in full. Plan ~8–12k lines per opus/high seat, and give a slice whose read list is a few giant files two seats rather than one. The area prompt states the budget and the large-file protocol (references/orchestration.md failure mode 18).
The generator. The slice rules are hand-authored, the files are scripted — two scripts under <scratch>/review-all/<run_id>/, kept with the run's intermediates (the first dark-factory run's pair is the shape to lift, not code to reuse: its paths and rules are that run's):
- Build the full slice records from
snapshot.json, the fix-density counts, the Phase 0 prior set and the leads: an ordered list of(seat, area, path predicate)rules, applied first-match per source file, with a last catch-all rule per area so no file is unassigned (the script refuses a file no rule covers); asub_area → seatmap for priors whose anchor file is not in the tree; per file the snapshot record's measures; the ranking above as a sum of percentile ranks, with the rotation term;read_fullycut at the budget; priors assigned to the slice holding their anchor file, else their sub-area's seat, else the area's first slice; leads routed byfix_wherepaths, else keyword rules, capped per seat. It prints the roster (seat, files, lines, read-fully count and lines, index count, tests, alarms, priors, leads) and the unassigned priors for you to check before anything is launched. - Write the seat briefs and the slim args: one
<scratch>/seats/<seat>/brief.mdper slice in the orderreferences/orchestration.md§The slice brief fixes,<scratch>/prior_open.jsonwith every prior in full, andargs.jsonin theargsshape — each slice reduced toseat,area,brief,read_fullyas path + lines, counts,size_alarms,prior_openas identities, andpriortostanding+full_path. Measured: 820 KB with everything inline, 60 KB slim; the inline form cannot be emitted as a tool-call value.
Phase 2 — stale instruments in delta mode
Run as sibling skill invocations (in this session, or spawned per skills/spawn/SKILL.md so they overlap), each against the same pinned tree ($TREE), with Phase 0's area rule written into its invocation line (the rule the record keeps as extra.area_rules), and in findings-only mode, so the inner runs file no tickets and no trigger chain — this run's Phase 5 triages the merged findings and Phase 6 files the one chain:
/hotspot-survey --refresh --as-of $AS_OF --findings-onlywhen stale — re-verifies its open findings against the pinned tree and writes its findings artefact under the contract./review --as-of $AS_OF --since <previous review as_of_sha> --findings-only --launched-from none— its delta mode.
Wait for both artefacts, add their run_ids to inputs_consumed, and fold their findings into the prior open set (Phase 1's generator writes it per slice and to prior_open.json) / leads by canonical. Their mechanical candidates are kept for the Phase 5 merge by key. The census is consumed only (its codebook entries carry finding_key; entries without one become leads).
An instrument that has never run under the contract on this project is offered in full at the cost gate — /hotspot-survey whole, /review --since none — not in delta mode. A decline does not remove it from the deliberation's inputs: the gap-fill runs before the Phase 6 sitting, never after close-out. Measured on reify (review-all-reify-20261010): the gap-fill run after close-out found 188 findings (18 high), 8 contradictions with program rulings and a fail-open gate hole, and forced a second sitting.
Phases 3–4 — the workflow
Pass the slim args.json Phase 1's generator wrote (the args shape in references/orchestration.md; seat briefs and prior_open.json stay on disk) and launch the script there through the Workflow tool (background), passing the parsed JSON as args. Routing is explicit on every seat; the session model is never inherited:
| Seat | Count (first DF run) | Model | Effort |
|---|---|---|---|
| area review | one per slice (20 on the first run) | opus | high |
| skeptic | one per seat (23 on the first run) | sonnet | high |
| calibration skeptic | 10, only when the area skeptics' refutation rate is under 0.05 and more than ten findings survive | sonnet | high |
| high-stratum skeptic | one per remaining confirmed high, only when the calibration sample drifts (≥ 30 % disputed or ≥ 2 sampled highs) |
sonnet | high |
| cross-area lens | 4 (overlay may add) | opus | high |
| synthesis | 1 | fable, else opus / xhigh | high |
| critic | 1 | fable, else opus / xhigh | high |
Fable provenance follows skills/team/SKILL.md §Fable: a fable lead routes the two seats; an opus lead needs --fable, a statement in the conversation or the overlay's standing ruling, and otherwise runs the fallback and says so in cost. Hard rules, each bought with a failure the hotspot skill records: semantic validation of every seat with one re-run (cleanSeat/degenerateSeat), pipeline() from each seat into its skeptic, skeptics blinded to proposals, a barrier only before the cross lenses, digest before synthesis, three verdict classes counted separately, log() for everything dropped. A refutation rate under 0.05 from the area skeptics — refuted / (confirmed + weakened + refuted) < 0.05 with n > 10, not only zero — starts the script's Calibrate phase: a stratified sample of ten confirmed findings — every high first, then the rest — each re-verified by a fresh skeptic with per-measurement falsification, beside the cross lenses, returned as yellow_flag_sample. A sampled finding the fresh skeptic weakens or refutes is recorded beside the finding as calibration_disputes_verdict (the critic_disputes_severity pattern: one skeptic overriding another needs a record, and the human rules; references/report-format.md §Artefact 2), and a weakened share of ≥ 30 % of the sample, or ≥ 2 of the sampled highs, is the drift signal the sample exists for: the whole high stratum is re-verified (calibration.high_stratum_reverified). Measured: reify 5 of 10 sampled highs weakened and nothing consumed it; solar_challenge_platform 2 of 10. The skeptics' verdict notes reach the synthesis and the critic, and the mechanical join checks — a key in two streams, duplicate keys in a matrix cell, a high with no stream — run in the script, not in the critic (references/orchestration.md §The script). You read the sample's outcomes, not the code (§Lead context budget), and the digest copies it to the report's extra.yellow_flag_sample; either way compare the result's calibration.refutation_rate with the previous runs' verification.refutation_rate and say in ## Method whether the skeptic is drifting. (First run: 536 findings across this run and its two Phase 2 siblings, 0 refuted; five sampled, five reproduced — a sample that small cannot tell "all real" from "skeptic too soft", which is why the skeptic now records a falsification attempt per measurement, the sample is ten, and seats run it.) Fallback when the Workflow tool is unavailable: Agent-tool fan-out with the team-high / team-medium seat types and the schemas as prose; pipelining and the journal are lost and the report says so.
Lead context budget
The lead's context is the run's scarcest resource: the first dark-factory run reached ≈850k lead tokens, and reify carries about three times its code. So the lead never reads seat prose; it reads digests and summaries that scripts produce — the generator's roster, the Phase 5 digest of workflow-result.json, the rendered report. Calibration is run by seats (the Calibrate phase), filing with more than ~10 candidates by one seat (Phase 5), and the digest and the renders are scripts.
The budget extends to Phase 6: the lead reads digests and rosters, never return JSON or full hand-backs; an author's hand-back reaches it as the questions it must rule on. Measured on solar_challenge_platform: the lead entered Phase 6 at 526k and died at 970k with four returns in flight; a fresh session's resume cost 203k to re-read state. §Lead handoff is the remedy.
Phase 5 — digest, keys, report, mechanical filings (inline)
The result is 300–600 KB and arrives truncated; write it to <scratch>/review-all/<run_id>/workflow-result.json and digest with a script, never by eye. The script has already mapped the synthesis's short ids back to canonicals and returned short_ids for the prose that still uses them, and severity_corrections with each one's acceptance — the digest applies the accepted ones to the findings record and flags the rest (references/report-format.md §Artefact 2); it never expands an abbreviation by hand. The digest mints keys by importing the one key implementation — shared/src/shared/finding_key.py once it lands, until then skills/hotspot-survey/scripts/findings_artefact.py (finding_key, normalise_anchor) — never a reimplementation, and the report states which it used; for every finding, prior verdict and refutation it sets first_seen (the run that first recorded the key, from the earliest artefact carrying it), appends this run_id to the prior's last_seen list on re-observation, fills supersedes from supersedes_canonical, carries sub_area, and applies the observation classes (including unchecked). It writes the JSON record and renders the markdown from it (its ## Method opens with the contract §5 yaml block; the markdown is a ~40 KB digest, and the per-area finding lists go to review-all-<project_id>-<YYYYMMDD>[-<n>].findings/<area>.md (references/report-format.md §Artefact 1), rendered by the same script — measured: all three reports after the first exceeded the old 60 KB cap, at 430 / 176 / 157 KB), all named by run_id, per references/report-format.md, into the overlay's output directory (default plans/). Copy <scratch>/review-all/<run_id>/snapshot.json to plans/quality-metrics/<run_id>.json, then commit the JSON, the markdown, its .findings/ files and that copied snapshot with git commit --only <paths> (docs-only: pre-commit skips pyright). Then remove $TREE: git -C "$ROOT" worktree remove "$TREE", or rm -rf "$TREE" when §Dropped and degraded records the archive substitution.
Filing (skipped under --no-file): the mechanical candidates of this run and of the Phase 2 instruments are merged by key first (first run: 75 — 30 from this run, 35 from /review, 10 from the hotspot refresh — each row recording its sources), then mechanical findings with verdict confirmed — and weakened ones only after you have read the skeptic's note and still agree the cost is real — go through the contract §8 protocol in order — key lookup via find_tasks_by_metadata(project_root, key="x_finding_key", value=<key>) (task 6385, live), which matches a list-valued key by membership, where a live task is filed:<id>, a cancelled task carrying x_acceptance_reason is accepted:<id> and one without it is absent; then search_tasks at score_threshold ≥ 0.6 confirmed with get_task; then submit_task (curator path, not planning_mode) with priority = severity, metadata.x_finding_key, x_finding_run, source="review-all", files file-level. Record the deciding step per finding in the report's Mechanical filings table; a ticket id is recorded as such until resolve_ticket names the task. Structural findings are filed by nobody here (contract §6). A live correctness defect (wrong behaviour in landed code today, not a cost of the next change) does not wait for deliberation whatever the class of the finding that exposed it: the digest records the defect as its own mechanical finding — its anchor, the behaviour, the fix that stops it — which goes through the §8 protocol above at this phase, while a structural finding behind it still goes to deliberation, carrying the defect's key; the program doc's ## Triage briefing carries a line naming the defect and its task. Store unreachable → file nothing, say so, dispositions stay open.
On a project's first run under the contract, before step 1, run a one-off x_finding_key backfill over the pre-contract tasks whose finding lives only in prose, so the key lookup sees them (measured on solar_challenge: tasks 284 and 517). A brief that runs step 2 carries a "known adjacent" list — the search_tasks hits at 0.60–0.62 already found not to be duplicates — so they are not re-adjudicated.
With more than ~10 candidates, hand filing to one seat (sonnet / medium) rather than doing it inline: it runs the contract §8 protocol above by script and returns a filings.json (per key: the deciding step and the task or ticket id) for the lead to read. The script is an async client over scripts/migrate_metadata_modules_to_files.py::FusedMemoryClient with call_tool_with_retry and write_failure_reason, in two passes — a search pass writing the search_tasks hits per key for the seat to decide step 2 on, then a submit pass that files the undecided keys and resolves their tickets. resolve_ticket takes ticket (not a task id) and 1–2 min each, with ReadTimeout retries; the first run's 60 tickets took about 50 min, so run the submit pass detached and poll its filings file, and pass a per-call timeout (180 s) to resolve_ticket rather than leaning on the client's 30 s default. A submit_task that times out may still have created the task (solar_challenge_platform's 408): reconcile-before-resubmit — repeat the key lookup (find_tasks_by_metadata on x_finding_key) and resolve the ticket before any second submit_task, and never resubmit on the timeout alone.
Lead handoff
Phase 6 starts as a fresh session by default, anchored on the committed report and record. At the Phase 5 commit the phases 0–5 lead writes HANDOFF.md into the program's state home ~/.claude/review-all/<run_id>/ and ends its turn proposing the sitting; the deliberation and the program run from a new session (spawned per skills/spawn/SKILL.md, or opened by the human). HANDOFF.md carries the sources of truth (report, record, metrics snapshot, as_of_sha, how the program's tasks are read from the store), the state (what is committed, what is in flight), the duties the successor inherits (the --launched-from escalation, the trigger chain, --yes and fable provenance) and the remaining work (agenda items, streams not yet authored or decomposed). At the Phase 5 commit no program state exists yet, so this first handoff is written by hand from the report and record; a Phase 6 lead handing off mid-program renders it with program_doc.py --handoff once the program-tooling PRD lands, by hand until then.
A Phase 6 lead hands off again at a threshold, not at the limit: ~700k lead tokens — a guess; the limit is not in the data. --resume <run_id> in Phase 6 reads the state home, or the committed .state/ snapshot (<program doc>.state/: program.json, rulings.jsonl, ledger.jsonl, written by the program-tooling PRD's renderer beside the doc and committed with it), and the task store, diffs them, and continues from the gaps. Freeze the skill for the run: the Phase 5 record carries the sha noted in Phase 0 in ## Method (references/report-format.md §Artefact 1) as extra.skill_sha, and every later session, a handoff successor included, reads the skill at that commit rather than at a moved main — git -C "$SKILL_ROOT" show <skill_sha>:skills/review-all/SKILL.md, and the same for references/ — a duty HANDOFF.md lists. A brief never points at a fleet result.md: the registry reaps it (reify: records gone in 1–2 days).
Phase 6 — deliberation, program, /prd, trigger chain
Close the run turn by writing the handoff and proposing deliberation with the report path as the anchor (§Lead handoff); the fresh session then follows references/deliberation.md: contested items first in a fixed shape (findings, next change, options with trade-offs and long-term consequences, invariant candidates in checkable form; under --yes the recommendation is taken as the ruling), rulings recorded as they are made, the program doc per references/report-format.md §Artefact 3, /prd per stream (agent team for agent streams, spawned interactive sessions for spawn streams — both lead-sequenced: seats author, the lead rules and runs the critic and fold, seats decompose in seam order, and the lead commits each wave's docs once — references/deliberation.md §Commit cadence), and the contract §11 trigger chain as the program's last act, filed per skills/_shared/filing-the-trigger-chain.md §§0–5 with skill="review-all" (bare name, no slash) and this run_id (gate identity is metadata.trigger_chain). Review-all specifics: the gate is filed when the first /prd stream files and each later session adds its high/critical tasks as dependencies, commit_planning only once the FILED table is complete; an older open review-all chain is superseded first (§5), and when this run was launched from that chain's review_due escalation (--launched-from), §5 step 2 resolves it ran as <run_id>; a program that files no tasks files no chain and the report's extra says so (§0). A wave's docs changes are committed once, through the lead (references/deliberation.md §Launching /prd per stream).
Fan-out hygiene: an Agent-tool seat return under ~100 characters is degenerate — a rule for this phase's raw hand-backs, separate from the workflow's degenerateSeat (references/orchestration.md), which judges an area seat's structured record — relaunch that seat once, then list the lane as lost; solar_challenge_platform had five limit-killed seats and 947k tokens of restart waste. Size a decompose wave to the remaining session allowance before launching it, so a usage-cap stop does not kill seats in flight.
Graceful degradation
| Missing | Behaviour |
|---|---|
review/briefing.yaml |
stop; /review-briefing first (areas are its keys; no second list) |
| hotspot baseline | no fix-history lane: run /hotspot-survey whole if the cost gate accepts, else proceed with leads from git log only and say so; the gap-fill still runs before the sitting (§Phase 2) |
/review report |
run /review --since none (whole) if accepted, else proceed; inputs_consumed says which; the gap-fill still runs before the sitting (§Phase 2) |
| codebook | census consumption skipped; no agent-transcripts evidence; say so |
radon / complexipy |
scripts/quality_metrics_snapshot.py exits 2 naming the tool and its pin; run uv sync --all-packages. If still absent, the run has no snapshot, metrics_summary says so, and nothing is hand-measured in its place |
| project is neither a Python workspace nor a single src-layout package | the Python snapshot cannot run; use the overlay's instrument (Rust: scripts/rust_metrics_snapshot.py); never run the Python default and call 2 % of the code the domain |
| overlay | generic mode: Python defaults, areas from the briefing, output under plans/ |
| fable not authorised | opus / xhigh on synthesis and critic; cost line says a fable seat was wanted and why |
| Workflow tool | Agent-tool fan-out with team-* seat types; no journal, no pipelining; report says so |
| Phase 2 siblings not run | fix-history and does-it-run evidence missing; neither heals by itself (no baseline, no trigger chain) — the gap-fill runs before the Phase 6 sitting, never after close-out (§Phase 2) |
| fused-memory / task store | file nothing (no tickets, no trigger chain); dispositions open; the next session files |
args too large for a tool-call value (every slice inline) |
never pass it; Phase 1's generator writes the seat briefs and prior_open.json to disk and a slim args (measured 820 KB → 60 KB); a run that cannot write scratch files cannot run |
a seat echoes a different seat/area than its slice |
the script overwrites both from the slice and logs the drift; the report's seat list is the slice list, and a join on the seat's own echo is never trusted (first run: 2 of 19 seats, 41 priors misreported unchecked) |
scripts/check_run_completion.py absent in the dark-factory checkout (the §2 probe; never the target project) |
skills/_shared/filing-the-trigger-chain.md §4 interim form instead of §3 |
| a seat degenerate twice | lane lost; listed under Dropped and degraded |
Never fail silently: the method header and §Dropped and degraded record exactly what ran.
Writing to memory
The report is the home of every finding, measurement and proposal (contract §12); memory gets pointers.
- At the end of Phase 5: one
observations_and_summariesentry — run id, as_of_sha, seats and cost, verification counts, areas with the highest-ranked cells, report and JSON paths — withproject_idandagent_idper the overlay andentities=naming the mechanical tasks filed ([]if none). - In Phase 6: one
decisions_and_rationaleentry per ruling,entities=naming the tasks or the gate id once they exist (references/deliberation.md§Recording). - Not written: findings, metrics, proposals, the matrix, seat prose.
Anti-triggers
- Does the software work after the orchestrator ran →
/review. - Where have bugs clustered →
/hotspot-survey(this skill consumes and refreshes it). - Confusion sightings →
/census(consumed, never run here). - One module →
/study; one bug →/deb. - Authoring remediation →
/prd, from the program doc. - Anything unattended: the human gate dispatches to a human or a spawned session; the instrument never runs from it on its own.
Reference
references/orchestration.md— the workflow template:argsshape, schemas, prompts, routing, failure modes, design notes.references/report-format.md— report, findings JSON and program-doc contract.references/deliberation.md— the deliberation protocol,/prdlaunch, trigger chain.references/project-overlay.md— overlay schema for specializing this skill per project.