Use when changing this repository's enforcement machinery — the deterministic gates, the validators, the build-runtime server's argument validation, the pure-leaf launch refusals and the read-only sandbox profile, the DEV hook layer — and whenever you are about to classify a review finding as 「残余」「到達不能」「対象外」 (residual / unreachable / out of scope). Required reading for fixing a fail-open, adding to an allowlist or denylist, touching the gate functions in `mcp_servers/build_runtime_server.py` / `tools/orchestration_runtime.py`, `tools/hooks/` (the DEV layer; the leaf hook layer, the capability gate and write_root derivation went in Z4, issue #171, and the episodes about them stay in `references/`), or the gates in `validate_pipeline_semantics.py`, for fixing an audit finding, and for triaging a subagent's or Codex's review findings.
Resources
2Install
npx skillscat add seiya/atmofab/atmofab-enforcement-change Install via the SkillsCat registry.
Changing atmofab's enforcement machinery
What this skill holds is the traps specific to the enforcement domain: dual-read pairs,
failure attribution, verification commands, and the judgment rules you never drop. How to
run a review — round structure, exclusion lists, when Codex enters, convergence, mutation
checking — is owned by atmofab-review-loop, so start that one too once you reach the review
stage.
Fixing enforcement machinery in this repository carries a high probability that the fix
itself introduces the next defect. In PR #51 (two high audit findings), 17 subagent review
rounds plus 3 Codex passes surfaced 15 defects that the fixes had introduced. What follows is
a procedure against those recurring shapes; it does not restate the rules themselves (the
canonical sources are mcp_servers/README.md / docs/HOOKS.md / docs/AGENT_SKILLS.md / docs/ORCHESTRATION.md §pure-function leaf —
copying them here would add one more twin document, which is itself the defect class this
skill exists to kill).
This file carries the rules; references/ carries the episode each rule came from. A rule
here that does not obviously apply to your case is answered in its reference file, not by
guessing.
references/judgment-episodes.md— what each judgment rule below cost, in fullreferences/input-surfaces.md— surfaces 5-13 (incl. 9-b and 9-c), the marker-narrowing version table, the recipesreferences/source-text-surface.md— the spelling variation a source-text-reading gate must survivereferences/dual-read-pairs.md— the table of facts two layers readreferences/failure-routing.md— attribution criteria, the known branches, and remedy wordingreferences/deterministic-substep-wiring.md— wiring a conductor in-process substepreferences/verification.md— the verification commands and the prose grep procedure
Judgment rules you never drop
These three hold everywhere in the procedure. Finish reading them before you open any
reference file. Episodes for all of them: references/judgment-episodes.md.
1. Before classifying anything as residual or unreachable, run the attack. Nothing goes into
the residual bucket without a record of an attempt that failed. Decide by what you ran, not by
who said it — PR #51's only P1 was accepted for five rounds on an unverified premise, after two
reviewers had found it.
1-b. Deleting a defense is also a classification. When you delete a defense you wrote, the
reason is always "this shape cannot occur", and that is the classification itself (PR #53 shipped
a fail-open that way, on the strength of one compiler probe).
- Do not conclude impossibility from a single counterexample — try at least one more
different spelling or different association path for the same construct - "The mutation survives" is not grounds for deletion. "There is no test" and "the code is
unnecessary" are different claims. If it survives, keep it and write in the docstring that it
is not pinned - This repo pushes the other way too ("delete dead defenses"). How the tug-of-war settles:
you may delete only when there is an execution record of an attempt to reach it that
failed. "No caller exists" (dead code) counts; "the language spec makes it impossible" does not - Before claiming "no record has reached it", name what ONE RUN of the mechanism LEAVES
BEHIND. The bullet above says what evidence counts and says nothing about how you obtain it,
and that gap is what issue #176 fell through: a census over the artifacts a mechanism writes
returned 0, and the mechanism does not mark them, so it would have returned 0 either way. It
was incapable of detecting its own subject, and the deletion shipped on it —write-step-result --backfillhad in fact run 7 times and carried an orchestration topass.
Readorigin/mainand name the trace concretely: theself.emit("<literal>")event name, the
file that gets created, the field that gets set. If the answer is "nothing", stop and claim
only what is provable — "no in-tree caller" — rather than "it was never reached". A pure
reader leaves nothing; so does a writer whose write sits insideif changed:, which sees
effective runs and is blind to the no-op the resume path actually took. Write "unreachable"
and "ineffective" as the different claims they are. Which artifacts are and are not evidence,
and the three follow-through traps (read the whole command line before classifying a hook hit;
resolve WHO called it; state each artifact's coverage window):references/verification.md§"Establishing that a mechanism was never reached" - The moment you write "the language spec makes this impossible" is the most dangerous one. If
you write it, execute one case from that spec and confirm. What breaks these claims is always
the language's special form (PEP 420 namespace packages, use association, implicit
association rules). Any "impossible" about package structure, scope, or visibility gets one run - DELETING A TEST CLASS is deleting every pin it carries, and a row is not classified by its
NAME. A class named after a mechanism you are deleting reads as "direct tests of deleted
functions", and the reason to delete it is the same "this shape cannot occur" as any other
defense — so it gets 1-b's classification, per ROW, before it goes. Issue #170 deletedUsageLimitTerminalLineStreamTestsunder that label; five of its rows pinned LIVE classifier
code in their FIRST assertions — fourteen mechanisms — and called the deleted function only
in their tails, and it took three review rounds (3 mechanisms, then 10, then 1) to restore
them, each time against mutants origin/main killed and HEAD did not. Rule: before deleting a test class, classify
each row by what its assertions OBSERVE — the names its body calls, read byast, not the
docstring or the class name — and keep the rows (or their first halves) whose subject
survives. The tell is a row whose docstring is about the deleted mechanism and whose body
starts with a call into something else. This is the test-side form of "what CHECK went with
what was deleted" (atmofab-review-loop§Stopping conditions), which otherwise fires only in
the disclosure round, i.e. two rounds too late. Episode:references/judgment-episodes.md
§Rule 1-b (issue #170) - MOVING a fact is deleting a defense you never classified, and it is the one shape here that
reaches no classification at all, because nothing prompts you to make one: the check still
runs, still passes, and now covers less. Issue #175 moved the adoptedprofileout of the
sidecar'sall_nodesinto a key of its own, and_closure_signature— the R6-lite freshness
comparison — readsall_nodes; so bumping a profile's version left every adopter byte-identical
at the new HEAD and CHANGED it onorigin/main, where the profile had been an ordinary node.
Rule: before moving a fact between structures, list the readers of the OLD structure and say
for each what it sees now. The tell is a diff that ADDS a key rather than changing a value —
nothing looks removed. Nothing in a review loop finds this except comparing the two revisions,
which is why it isatmofab-review-loop's disclosure axis that caught it and not the sweep, the
census, or a blank-slate reviewer, all of which compare HEAD against itself - Two more forms of the same loss, both from issue #316 PR-2 (PR #319), and neither looks like
a move of a FACT. (a) Text moved to a new FILE leaves every check that names files by
path. The OpenMP floor paragraph moved verbatim from a Fortran prompt fragment into a newparallel/openmp/one, and the whole-file allow-directive scan read a hand-listed tuple of
paths: a copyable directive planted in the paragraph was red onorigin/mainand green on the
branch. Rule: before moving text between files, list the checks that read the SOURCE file
(os.walkfor its path acrosstools/tests/), and derive their file set from a directory
rather than adding the new path — the derivation then needs its own coverage row, or a
narrowed derivation is silent (round 2 found three narrowing mutants surviving the first fix).
(b) Re-pinning a SHIPPED version in place deletes the revert check. A drift pin's old
digest is what the "no empty bump" check collides a revert against; overwriting it made
"revert the change under a new version" pass. Rule: when a digest that has shipped (it is onorigin/main) is re-pinned in place, keep the shipped one as a<version>@<main sha>entry,
aspure-50@3c117410does. The tell for both: the fix commit's diff changes a value or adds a
file, and every check stays green — only running the defect at both revisions shows it.
Episode:references/judgment-episodes.md§Rule 1-b (issue #316 PR-2)
1-c. Severity is a classification too. Do not decide it from one reproduction. Rule 1 says
decide "does it happen" by execution; it says nothing about how far it happens. The procedure
is: enumerate every place that reads that fact, then open each layer (PR #55 got one severity
wrong three times, each time having run a reproduction).
- Do not use a layer you have not executed in the verdict; say explicitly that you have not
executed it - Finding the counterparts:
references/dual-read-pairs.md. When a fact is not in that table,
that is exactly when you write the enumeration out - Once every reader is open, enumerate what the other layers already cover before writing the
severity. The number of readers alone does not justify high — and the correction runs both
ways (PR #57's fourth correction was toward the safe side: the defense was silently dead, so
the fix shipped, but "this is exploitable" was the wrong thing to write) - But never turn "another layer catches it" into a reason not to fix. The enumeration exists
to make the severity description accurate, not to decide whether to fix
1-d. The premises of a fix you have not written yet are also a classification. Rules 1
through 1-c are about things that already exist. If an unwritten fix's premise is false, the plan,
the implementation, the tests and the prose are wasted at once (issue #75: a six-phase plan died
to one unbilled observation).
- The moment your plan says "today this shape is handled like so", execute that one sentence
before implementing. Docs, old logs and issue bodies are not sources for a premise - Observation costs two orders of magnitude less than implementation
- "It was not refused" cannot be shown from refusal logs. In a layer where refusals leave no
event, count the traces on the success side - Before diagnosing a defect in enforcement machinery, run the suite FILE that covers it on
origin/mainand read what fails. The bug is often already a red test there, and that is the
strongest evidence a causal argument can have — but the FULL-suite verdict hides it behind the
known baseline failures, so the whole-tree number is not the place to look. Seconds to run
(references/verification.md§The suite) - Executing the premise on ONE layer is not executing it. A premise about a VALUE names
at least two layers — the one that ACCEPTS it and the one that PRODUCES it — and a run on the
accepting side alone answers a different question. Issue #235's plan verified "a string is a
scalar to the snapshot shape gate" with one call (_infer_json_shape("flat") == [], true) and
shipped a leaf-read remedy telling the compile leaf to route an enumerated input as a string
scalar — which no runner can emit (the snapshot getters return numbers). The fix reintroduced
the class it fixed: a compile-side contract naming a capability with no producer. Before
writing a remedy or a contract sentence, name the layer that PRODUCES the thing and run
that one too (references/judgment-episodes.md§Rule 1-d, issue #235) - A premise about what a DRIVER does "for every node" is a claim about that loop's SKIP
branch, and the skip branch is the one a read of the loop body does not show you. Issue
#238's plan rested decision 4 on "--with-depsrunscheck-phase-certifiedfor every closure
node, where the new clause applies"; the closure driver asks_dependency_node_readinessfirst
andcontinues past a ready member without launching a conductor, so the clause never ran on
a dependency — and every member of the real closure was ready while failing the current
validator. Five remedy texts shipped pointing the operator at a command that skips the thing
they were told it re-certifies. Rule: name the member the driver SKIPS, and run the driver
on it (references/judgment-episodes.md§Rule 1-d, issue #238) - When a premise collapses, keep the measurements. The plan dies; the measured facts stay
1-e. "Out of scope because the leaf gains nothing" is a classification rule 1 does not govern.
Rule 1 answers can this happen and demands an execution record, which is right for
unreachable — a claim about the mechanism. It is the wrong rule, and an expensive one, for
out of scope, a claim about the premises: AGENTS.md §"Development premises" defends against aleaf shortcut, an action getting a leaf closer to reporting its task done without earning it, and
against nothing a leaf gains nothing from. A finding on the second side is dropped by naming what
the leaf would gain and showing it is nothing, with no attack run.
- Confusing the two is a defect in both directions. Say "unreachable" about something you never
attacked and PR #51's P1 comes back — five rounds in the residual bucket on an unverified
premise. Demand the attack for a gains-nothing finding and you bought a reproduction for
something you were dropping either way. Decide which claim you are making before you decide
what evidence it needs - The test is what the leaf is closer to afterwards. Write "a leaf that takes this is closer to
___". If the blank fills with reporting asubstepdone — a check passed, a gate cleared, a
verdict green — rule 1 applies and you run it. If it cannot be filled, it is out of scope - Intent is not a defense. A leaf under a goal has a motive to find the spelling that works, so
"no leaf would phrase it that way" carries no weight on a finding that reaches done - This deletes no existing defense. Deleting is rule 1-b's question and needs 1-b's evidence.
What this rule decides is whether you ADD one, and whether a finding against one is worth a round
2. Do not close an environment-dependent finding with a mock on the test side. When told
"this test fails on a machine without the compiler", first ask what happens in production on
that machine (PR #51: mocking which capped a hole where the rule was inert on machines with
no compiler installed).
3. Changing a rule is not done until you have swept the prose that cites that rule as grounds.
Use the grep procedure in references/verification.md. Right after you write a sentence,
execute it: numbers, rule ids, compiler diagnostics and "X catches this" are all executable
claims, and write a range when the number varies; executing it is not enough if the probe was
contaminated — a compiler writes artifacts beside its input and reads them back, so a shape can
be answered by what the PREVIOUS shape left, and a cleanup line between two probes makes one
recorded row into two environments (issue #153 recorded a gfortran rc=0 that holds only with a
stale .smod present, beside a rc=1 taken after the rm; references/verification.md
§"Verification steps that silently do not run" carries the reproduction). The contaminant need
not be the probe's own history: a suite running CONCURRENTLY can be swapping the file the probe
reads — issue #180 ran test_backend_boundary --check-baseline while the suite was in flight
and read the one-entry baseline test_write_baseline_actually_writes_the_measurement installed
at the REAL path for the length of one try (that row now writes a temporary copy; the class stays), getting every scanned file reported as growth from
zero, with git status clean throughout because the test restores the bytes in its finally. Run the verification set with nothing
else in flight, and say so beside the number. And the flip side of rule 3 is that prose you newly write in the same commit is
unverified until you run it (L128 got four freshly written measurements or citations wrong
inside the fix itself). Do not write someone else's measurement as your own — cite the source
explicitly, or re-measure before writing. Arithmetic on an inherited count is that
substitution in its quietest form: deleting one member and turning "all three" into "both"
carries the old, unverified count forward as if you had measured it (issue #178 — the "three"
had been six, so the "both" was five; references/judgment-episodes.md §Rule 3). The place this fires that you will not expect is a
CORRECTION: a reviewer's finding arrives already carrying evidence, so it feels verified before you
write it, and issue #153 wrote a reviewer's HEAD-measured count into a corrections bullet as the count
"when that commit landed" — replacing a correct figure with a wrong one, one item after condemning
that exact substitution. Re-measure at the commit you are about to NAME, not at HEAD
(atmofab-review-loop's "the fix was to a RECORD" sign owns the procedure).
Keep a list of every place you wrote a number and
re-measure them together at the end; a commit message cannot be fixed afterwards, so either
mark the number as measured at a point or rewrite it in the final round.
A new EXCEPTION to a prohibition is a change of that rule, and it must reach every statement
of the prohibition, not only the gate. On a leaf-read surface a prohibition left absolute
beside a text that requires the excepted act makes declining the task the leaf's cheapest
resolution, and the reviewer's own inlined contract then reads as endorsing the decline: issue
#316 PR-4 relaxed "a physics source never uses the harness" in the checks-source gate and the
MPI fragment, and left it absolute in the Fortran fragment's rule (a), the neutral checks
contract §4 and the Fortran binding §4 — two documents on the reviewer's transport, one on the
producer's — until round 1 found the contradiction by rendering both prompts.
3-a. When the sweep keeps losing, COUPLE the documents to the rule with a check. Rule 3 is a
discipline, and it failed four consecutive rounds on issue #71 after it had been diagnosed
(carried, not found: the narrowing round discovered no record defect of its own, so "four rounds
found it" would overstate the history) —
the worst instance told every leaf, in the one document every leaf reads, that a refusal it can
actually receive cannot happen. The trigger is the count; the audience is the priority: three
or more statement sites is when discipline has already lost, and a site read by a leaf or an
operator does not lower the count — it decides which site you check first.
Reach for the pattern this repository already uses — but the instances are DIFFERENT
shapes, so read the one nearest your rule and treat copying as the starting point. The live
ones (the three in tools/tests/test_hooks_cli.py that this paragraph used to name went with
the leaf hook layer in Z4, issue #171 PR-1; their episode stays inreferences/judgment-episodes.md §Rule 3-a): tools/tests/test_pure_leaf_wiring.pytest_phase_02_states_the_scope_rule_with_its_bound (anchor on text that precedes the rule,
byte-identical in the refused wording, plus the refused spelling asserted absent) and_RUBRIC_POINTER_SURFACES (the surface tuple asserted as a literal first, the anchor's count
pinned to one, the reader bounded to that line); andtools/tests/test_verdict_evaluator.py::CheckRefLeafStatementSitesTest (issue #269: an
anchored window with a self-tested bound, a statement reader that takes a hard-wrapped
continuation and stops at the next item, the token DERIVED from the constant, the refused
spellings asserted absent inside the window rather than tree-wide because one of them is a
legitimate vocabulary elsewhere, and a PRODUCER surface beside the reader surfaces so the
check is red in both directions). The eleven traps, each of which cost a round (the count has now been wrong here TWICE, both times by addition and both times caught late: "four" over five bullets until issue #143 added the sixth, which is this rule's own enumeration trap turned on itself; then "ten" over eleven, when issue #180 added the STATEMENTS-per-site bullet and did not re-count — the same trap, a third time, in the same sentence. Issue #175 added four of them, all found by a witness census run against a coupling check written the SAME DAY, three demonstrated by planting the defect the check was built to refuse. Re-count this sentence whenever you add a bullet; nothing compares it to the list.):
Anchor on text that PRECEDES the rule and is byte-identical in the wording you are refusing.
Anchoring on your own corrected sentence pins that the correction survived, not that the rule is
stated — witness the check by restoring the old wording and confirming the failure names
what is missing, not the anchorBound the reader and self-test the bound, or a document that mentions the rule's terms
anywhere passes on the strength of an unrelated sentenceDecide what "names the rule" means for THIS rule. Couple by MEMBERS only when the prose
names them in full; otherwise couple by POINTER (each site cites where the constant lives) or by
NUMBER. The rule is defined once, IN THE CODE, and the documents are checked against it —
never the reverse, and never both spelled out independentlyPin the members, not the source line. A legitimate extraction to a named constant must not
turn a true statement red — resolve a named constant before giving up, and make the failure name
the repair. This is the easiest one to reintroduce, because pinning the spelling is three lines
and pinning the members is fifteenWhen the rule is an ENUMERATION, couple element by element AT ITS STATEMENT POSITION, and
derive the count from the code. "Does this token appear in the document" couples nothing the
moment the token appears for another reason — on PR #116 the check asserted each of three flags
appeared in the canonical document, and a reviewer deleted a whole channel bullet with the row
green, because the flag names also occur in that document's reproduce command and in an
unrelated bullet. What closed it was reading the section that HOLDS the enumeration and
requiring each element to OPEN its own bullet there. The count is the other half: a document
that says "three channels" and lists two is only wrong if something compares the two, and the
number belongs to the codeCount the STATEMENTS per site, not the sites — one rule is routinely stated twice in one
place, in two different FORMS, and coupling either one leaves the other free. The bullet
above is about the ELEMENTS of an enumeration; this is about the same rule said a second way
beside the first, which does not look like a second site at all. Issue #180 coupledvalidate_pipeline_semantics's--stageset to the validator's argparsechoicesat both of
its restatements — and each restatement says the set TWICE: as the alternation'stage': 'a|b|c', and as a prose qualifier naming which stage takesir_ref
((compile stage)). The check read the alternation only, so setting BOTH qualifiers to(plan stage)—planbeing the non-existent stage whose appearance in that help was the
finding the check was written for — left the file green. The fix is not a wider regex: it is
to find the CODE that answers the second form too (there, driving each declared stage and
reading which one's violation saysrequires non-empty --ir-ref) and couple it separately,
self-testing the derivation so a reworded answer fails the row instead of silently answering a
different question. The tell: a restatement that reads as a sentence rather than a list —
the qualifier, the "(defaults to …)", the "…, which is what the gate runs" — sitting inside the
same cell as the machine-shaped form you coupledAn EXEMPTION is a rule too, and it gets broken from both sides. When the check must let
something through — a routing sentence that legitimately names the thing you are refusing —
do not decide the exemption by pattern. Issue #143 tried "the line cites the phase doc", then
"cites it WITH the section pointer", and one review round broke both directions at once: a
hand-assignment written WITH the pointer passed, while correct routing prose about another
phase's rule was refused with a message calling it an assignment, whose only satisfying edit
made the document wrong. Deciding "routes" from "assigns" by pattern is §1's source-text
surface asked of an exemption, and it loses the same way. What closed it: an explicit
allowlist of the exempt lines, compared as a set — a new mention of any wording is refused
until someone reads it and adds an entry, which is a review gate rather than a judgment, and
adding one is a line. Two follow-through traps, both hit on the same branch: normalize the
allowlist entry to the WHOLE line (a 60-character prefix let an assignment ride on an
allowlisted line's tail — a new mention, which the check advertised as always red — so each
entry carries a digest of the full line), and an emptiness set does not self-test its own
surface list: deleting a surface that happens to carry no exempt line is invisible, which was
3 of 5 there, so derive the surfaces from the code that decides them
(leaf_contract_doc_refs) and assert the coverageRead the STATEMENT, not the line — prose WRAPS. A reader that tests one line at a time
misses a rule written across two, and hard-wrapped Markdown and prompt templates are what these
surfaces are made of: issue #175's census wrote the refused fact over two lines and the check
passed it. Span a bounded window of CONTINUATION lines — not across a bullet, heading or fence
boundary, or two adjacent items joined by chance become a statement nobody wrote — report the
innermost match so a one-line statement keeps a one-line key, and witness BOTH directionsKey the allowlist by SURFACE and content, not content alone. A digest-only key makes
copying an allowlisted sentence VERBATIM into another document free — which is precisely the
operation that produced the copies you are coupling against, so the check's whole purchase is
void for it (planted and green on issue #175). It also blinds the set comparison in the other
direction: with one line in two files, deleting it from one raises nothingThe set comparison's DISAPPEARANCE half needs its own witness. "Compared as a set" is a
claim about two directions, and only one of them is exercised by every run — replacing the
stale computation with[]stays green forever. Drive it with an entry that matches nothingDeriving the surfaces is not enough if the derivation SWALLOWS a miss. A
.is_file()guard
over a derived path turns a renamed or moved file into a SMALLER scan, silently — the census
renamed aSKILLand planted a false statement in the file the rename orphaned, and the check
passed. A surface the code NAMES but the tree does not carry is a failure of its own row, and
each group's expected count comes from the same source the group does
Before adding a check, ask whether the sites should exist. The cheaper fix is this
repository's ordinary practice — one canonical statement, everyone else cites it (AGENTS.md
§Dedicated rule documents) — and it cannot rot. Coupling is for the sites that must repeat the
rule anyway: a leaf-read contract has to be self-contained, and a refusal message has to say it to
whoever was refused. (This is NOT surface 5's twin, though both count to three: surface 5 changes
the CHANNEL a decision travels on; 3-a keeps many statements of one rule honest.)
Sites a test cannot reach are real and the check does not cover them: a commit message, and a
prompt assembled at runtime that nothing renders under test. For those the only moves are to
remove the statement or to make it derived; say in the commit which sites you could not couple.
Check before assuming a site is out of reach — an issue or PR body can be edited
(gh issue edit --body), and docs/examples/*.yaml is coupled today bytools/tests/test_llm_config.py. THIS file is not: no test checks that a path cited in a skill document is tracked (#183 retired test_skill_citations.py), so check a cited path with git ls-files --error-unmatch -- <path> before you commit. Resolve it against the right base or it reports a correct pointer as missing, which is the failure that gets a check disregarded: a pointer starting references/ or scripts/ is skill-relative (the form a reader follows most often — the retired test resolved these); one starting with another skill's NAME, as atmofab-review-loop/references/mutation-testing.md does, resolves against .claude/skills/ (the retired test emitted no target at all for these, so here the manual check is stricter than what it replaces); everything else is repository-root relative. Both non-root forms occur in this tree today. And this check answers only half of what was retired: test_skill_citations.py also refused a file under a skill that no SKILL.md reaches by any pointer, and no command finds an orphan — for that, walk the skill yourself when you add a references/ file. What stays uncoupled is the CLAIM a sentence makes, not the paths it cites. A runtime-assembled prompt stops being out of reach the moment
you render it through the production entry point with a fixed payload: issue #149's sweep drivesbuild_launch_request -> prepare_launch_request_payload -> render_launch_prompt_text over every(step, substep) an LLM leaf runs, in every renderer shape, and scans the render minus the lines a
template file already carries. What then stays out of reach is not "a prompt" but a branch the
payload does not drive — which is why that sweep wraps each builder and requires it to EMIT under
some configuration, and why it needs a separate witness for the case it structurally cannot see: a
renderer that stops CALLING a builder drops out of both sides of a reach equality derived from the
calls that happen.
Procedure
1. Inventory the surface before fixing
Enumerate, at the argument-name level, every caller-supplied input that reaches exec / env /
argv / the filesystem / the paths from which the gate reads its evidence. "Plugged one, the
neighbour was open" is the most frequent recurrence (env allowlisted → argv was open → the
value was open → the auto-discovery path was open: four in a row).
Leave the enumeration in the commit message or the pull request. The next round's reviewer will come
looking to break it.
When the gate reads the source text rather than the meaning of an input (validators and
parsers), the surface is a different one — it is the spelling variation the language
permits, and writing out each grammar is the losing line. Close the family at once by
inverting the polarity: do not parse statements that start with a keyword; take every identifier
that appears in them to the safe side. The concrete spellings (keywords are not reserved words,
the optional space in a two-word keyword, omitted ::, statement labels, the 18 attribute-bearing
forms) and the boundary note that governs them are in references/source-text-surface.md.
When a rule derives its safety from an enumeration, write a test that kills each element of the
enumeration by mutation (round 0 in atmofab-review-loop). A missing element shows up in no
other test.
The surfaces that are none of exec / env / argv / FS / evidence paths and none of the
spelling variation (numbered 5 through 13, with a 9-b and a 9-c). Each is one question; the episodes, the version tables and the measurement
recipes are in references/input-surfaces.md:
- Surface 5 — is caller-controlled data mixed into the classification channel the verdict
reads? If a marker string,failure_category, reason or excerpt is obtained by scanning
output text, and that text embeds data whose content a caller decides (file names, identifiers,
paths the leaf chose), the classification is forgeable. L174 was broken three rounds running
and each narrowing was defeated by one byte. Rule: change the channel rather than narrowing
the sample — exit code, exception type, a dedicated field, a sidecar; each is written by the
side that knows and cannot be written by the caller. This is the classification-channel version
ofatmofab-review-loop's sign "the pin was broken in a different shape every time — move the
definition to one place". Then count every site that makes the same
decision (there were six, and two further sites scanned neither marker), and at three or more
change the channel design instead of fixing them individually. Two follow-through traps: a
type channel dies silently on a list rebuild ([str(v) for v in …], a JSON round-trip, asorted(...)) so trace the container from emit site to decision and witness it through the
real CLI in a real subprocess; and counting the sites is not answering them — a site you
leave unchanged needs a comment saying it is deliberate, with unreachability proved by call-graph
closure rather than by reading. A third, when the party you are protecting the channel from is
the program being EVALUATED: the emitter is on the channel too. Moving a status off files the
program can write onto a stream it cannot is half the job if the script that PRINTS the status
is itself a file the program can write — bash executes a rewritten script file's rest — and so
is any input a later step runs (a control file the next command reads). Enumerate what the
evaluated program can write before the host reads the record: the record's channel, its emitter,
and every artifact the record names (references/input-surfaces.md§Surface 5, issue #293) - Surface 6 — if the check tells the reader "do this to fix it", what else does that remedy
rewrite? Wherever a ratchet, baseline or allowlist says "on failure, run this to update",
check that the command does not also rewrite the pin (PR #66: a pin described in three places as
unremovable was regenerated by following the instructions, then recurred in three stages as
each fix left a wider thing washable). Rule: never let a pin and the command with authority to
loosen it live in the same file or the same procedure, and pin the rule, not the result the
rule produced. A remedy is also READ — the two rules for that (order the causes by reachability;
never let it be followable by half) are in §3, because they are about the message a failure hands
back. Open both when you write one - Surface 7 — when you say you closed a configuration surface, what else does that tool read?
A flag that narrows configuration sources (--setting-sources,--config) governs
configuration files and nothing else; auto-injected memory, environment variables,
cwd-discovered files and startup fetches stay outside it (PR #72: the operator'sMEMORY.mdkept
reaching the leaf's first user message under a flag documented as closing the surface).
Do not conclude from the flag — capture what the process actually received and count it
(pointANTHROPIC_BASE_URLat a loopback server, save the request body, return 400: unbilled,
and the same harness reads back the permission layer's real verdicts if you return a synthetictool_use; mix in one control that must be refused, or "everything passed" cannot be told
from a dead layer). When several paths claim closure, observe each path — one closed for
another reason structurally hides the hole in the other. Take a cleangit worktreeas a
control, because a development checkout's untrackedsettings.local.jsonis a confounder, and
do not decide from one entry. A closure claim names the surface it closed — not "closed
the configuration surface" but "closed the configuration-file surface" - Surface 8 — the place you created for isolation is an input to that tool. Create a private
configuration dir to confine a tool, make it writable, and everything the tool reads from that
dir is a new injection surface (issue #63: one home shared by every leaf meant agenerateleaf
could write instructions the latervalidate.judgeleaf reads — a forged certification with
zero trace in the artifacts). Enumerate what the tool READS from that dir separately from
what it WRITES — the write enumeration is about availability, the read enumeration about
security. Bind the dir ro and allowlist only the writable places, so unknown names added by
the tool's next version fall on the inert side, and build that allowlist by measurement with
the kind of leaf production runs (measured with a tool-less leaf it missed 2 of 6).
Measure and state the cost of the polarity, and decide shared vs per-leaf first — the
severity came entirely from sharing - Surface 9 — "it moved under something the other side already covers" is a claim about a
CONFIGURATION, and it holds only for the configurations you enumerated (PR #86: the covering
root is relocatable by an environment variable, and with it moved a leaf could read a sibling
orchestration's transcript, while three documents asserted the closure unconditionally).
Enumerate what can MOVE the thing before writing that it is covered: an environment
override, a config key, a CLI flag, a symlink, a different$HOME— each is one execution.
The fix that survives is one resolver, not one sentence: move the resolution into the module
that owns the protected-root list so the tree that gets created is the tree that gets guarded.
And attribution is not enforcement — roots are sorted longest-path-first, so a new entry
beneath an existing one silently takes over the block message; keep a control read that must
still be attributed to the root above - Surface 9-b — a HOST write that lands OUTSIDE every child window is a new thing for the
write-attribution layer to classify, and the layer has no category for it. Surface 9 asks
what can MOVE a protected path; this asks what happens when the same path is written at a new
TIME. HISTORICAL since Z4 (issue #171
PR-2): the write-attribution layer this surface is about is deleted, because apure-function leafhas no write authority to attribute — the host writes every artifact.
The surface is kept because the SHAPE recurs wherever one layer classifies by window and
another writes outside it. What it described:_validate_actual_write_pathsattributed by
WINDOW, a leaf's changes diffed against the baseline its launch took, and a child-authored
path excluded from the orchestration's own diff only while its digest still equalled what the
child left (_child_managed_paths_excludable_from_orchestration_diff). A host write after the
window closed broke that equality and the path re-entered the orchestration-role diff, where
the orchestration's write_roots were the ORCH ROOT ALONE — so it read as an unauthorized
write to a path the host is entitled to write. Issue #177 met this by stamping a
certification into a stage meta atwrite-step-resulttime; measured, the direct call DOES
flag it. It shipped because no production caller records a terminal orchestration-role agent
run —update_orchestration_statusrewrites that row in place — so the branch is
unreachable today, and the PR says so in both directions rather than only the convenient one.
Rule: before adding a host write to a path a leaf also writes, name the window it lands in.
If the answer is "none", drive the attribution layer for EVERY actor role, not just the
child's, and write down which roles are unreachable and why — "the check that would refuse
it never runs" is a statement about today's callers, and it is the sentence the next person
wiring that caller up needs to find. The cheap probe is the function itself with a synthetic
payload per role; it took minutes and it is the only reason the residual is documented instead
of latent. - Surface 9-c — a hide placed AT A PATH is defeated by every SECOND NAME of the thing it hides,
and the second names are not enumerable by hand. Surface 9 asks what can MOVE a protected
thing; this asks what else the thing is CALLED.render_bwrap_commandhid the artifact trees
with tmpfs overlays at the checkout's path (one tmpfs over the WHOLE checkout since issue
#227, at the same path — the rule is unchanged, and what a second name now exposes istools/
as well), and issue #226 widened the read-only bind of the
backend CLI's install root frombin/to the whole$HOMEchild — so any second name for the
checkout inside that root exposed the producer'sdialogs/to a VERIFY leaf with nothing
overlaid. Five review rounds found five names of the same class, one per round, each fixed by
the comparison that closed the previous one: a..in the PATH entry (lexicalnormpath), a
symlinked$HOME(realpath), a bind mount (realpathis BLIND to it —os.path.samestatby
inode against the checkout's ancestors), the mirror — a$HOMEchild symlinked INTOworkspace/(the inode walk in both directions), and the mirror by bind mount
(/proc/self/mountinfo: a path's(device, path within the device)for the root and every
mount beneath it). Rule: when a defense is keyed on a path, compare IDENTITY, never a
spelling — inode for symlinks, the mount table for bind mounts, both directions (the root is
the thing, holds it, or lies inside it) — and write the exemption for the one alias that is
safe by construction (a root whose own spelling contains the thing: the later bind stacks on
top). The tells: a fix that addsnormpath/realpath/resolve()to a comparison (each is
one alias class, and the next round brings the next), and a docstring that says "the overlay
covers it" without saying at which spelling.atmofab-review-loop's "same family two rounds
running → change shape" fired at round 3 there and the shape still needed two more rounds,
because each identity instrument sees one alias class; enumerate the classes (lexical,
symlink, bind mount) FIRST and pick the instrument that sees all of them, or state which one
it does not see and why. When the thing is not a file at all — an installation, the
runtime a binary links against — no comparison of paths establishes it, identity or
spelling: ask it what it DOES. Issue #316 checked that an MPI launcher and compiler wrapper
were one installation by comparing their directories; Debian's alternatives put two
installations' programs in one/usr/bin. What replaced it builds a program with the one and
runs it under the other (references/input-surfaces.md§Surface 9-c, "an installation").
And enumerate the BINDS the check must cover from the code that
emits them, not from the check's name: issue #227 rewrote this rule in three consecutive
rounds — keyed on the exempt root, then on the mount point, then on where the checkout
APPEARS — and only the third round noticed the check had never run over the system
directories' recursive binds at all, because every prompt and docstring said "install
root". Episode:references/input-surfaces.md§Surface 9-c. - Surface 10 — when a gate delegates its verdict to an external tool, can that tool decline to do
the work and report SUCCESS? A gate that reads an exit status is trusting the tool to have
looked. Tools bound their own effort — a configuration cap, a time or memory limit, a walk that
skips what it cannot read, a parse it abandons — and the honest ones then say so on a channel
the gate is not listening to. PR #125:cppcheckanalyses at most 12 preprocessor
configurations and stops, exiting 0 over source it never analysed; the only trace is[toomanyconfigs], whose severity isinformation, which the declared severities did not
enable. A leaf that cannot satisfy a finding wraps the file in#ifdefs and the gate passes.
Rule: enumerate the tool's OWN limits from its--help, not its failure modes, and for each,
ask what it reports and where. Prefer removing the cap to raising it —--max-configs=Nonly
moves the hole toN+1,--forceremoves it — and check the degenerate case fails CLOSED (there,
a timeout arrives asreturn_code: None, which the conductor already refuses). The tell is a
tool that reports partial work as completion, and every family this repository delegates to has
one: a linter's walk exclusions and read errors, a compiler's error cap, a test runner's
collection error, a build system's-k. Related but different from surface 5 — nothing here is
caller-controlled; the tool is simply answering a narrower question than the gate asked. - Surface 11 — does a field the LEAF authors decide which gate applies to its own artifact?
Not what a gate reads to judge — what a gate reads to decide whether to judge at all. A gate that
resolves the artifact's kind, its language, its declared model, or the presence of a contract
section FROM THE ARTIFACT, and returns early when the value is one it does not cover, hands the
leaf the off switch. Issue #168:_validate_published_surfacetakesspec_kindfrom the IR's
ownmeta, and acomponentIR self-declaringproblempasses the whole deterministic compile
gate with rc 0 — measured on a real certified IR — dropping every published-surface pin. The
same field, twice more:_validate_toolchain_backend_supported(deleted with the IR's toolchain in R4-a PR-3, issue #284) read it to decide whether a
node was exempt from half its own check, and the phase contract warns that whichever of three
places a multi-dimensional state contract appears in FIRST is the one the gate validates — so an
empty mapping in the highest-priority place makes it validate nothing. This tree has also
CLOSED one instance already, and it is the shape to copy:docs/workflow/phases/phase_01_compile.mdrecords that thespec_kinddeciding theinfrastructuredependency-count exemption "is read fromspec_catalog.yaml, not from the
self-declareddeps.yamlfield". Rule: resolve the fact from the HOST'S OWN identity for
the node — thenode_key, the launch request, the reservation — never from the document under
review, and where a gate legitimately must read it from the document, pin the document's value
against the host's before the gate runs. The tell: an earlyreturnin a gate whose
condition is a field of the thing being gated. Enumerate those first; each one is a switch, and
the leaf holds it. Distinct from surface 5, where caller data pollutes a classification the host
computes — here the host asks the leaf a question and believes the answer. - Surface 12 — when you change how a leaf is DRIVEN, what stops being delivered to it? The
other eleven ask what an input decides. This one asks what a TRANSPORT carries, and it is
invisible from the diff: moving a leaf from the agentic loop to the pure one deletes itsskill_must_read_refswholesale, so every force-read document it had is silently cut, and
a rule that reached it only that way is now enforced by a gate and stated to nobody. Issue #169
cutdocs/workflow/CHECKS_MODULE_CONTRACT.md§5 — the deterministic lint/syntax rule set for
leaf-authored source — off the one leaf whose runner it was written for, whileleaf_contract_doc_refs' own docstring names that leaf as the reason not to gate the injection
on node shape. Rule: before changing a leaf's transport, enumerate what the OLD transport
delivered — the SKILL, eachleaf_contract_doc_refsentry, each must-read the request built —
and say for each where it goes now. Three answers are legitimate (inlined as context,
restated in the template, genuinely not needed) and a fourth is the defect (nowhere). The
tell is that nothing fails: the gate still runs, the suite is green, and the cost is a leaf
that cannot satisfy a rule it was never shown — a pass-rate defect that reads as model
weakness — and the count you quote has to say what it counted: measured at issue #169's round-3
commit, the leaf's prompt named NONE of the seven rule codes by code and carried three of the
seven obligations in prose. Where the cut document is BACKEND knowledge, restating it in the
template is the one optiondocs/BACKEND_BOUNDARY.mdforbids (a new neutral-core file starts
at zero, so any such token is growth); deliver it from the backend package through the registry
instead, which is where that rule says the knowledge already lives.
Nothing in this repository's review loop finds this except rendering the prompt — the
mutation sweep mutates code, the census enumerates code, a blank-slate reviewer reads code, and
all three were green on it; seeatmofab-review-loop's "what does a LEAF see" clause.
The MIRROR is the same surface and reads as an unrelated task: changing what a leaf is TOLD
without changing what it is GIVEN. Issue #175 corrected the agentic compile producer'sSKILLto say the direct dependency set is read from<ir_ref>/dependency_graph.json— and
leftbuild_launch_request's must-read set as it stood, so the one leaf that reads exactly its
required set had no source anywhere in it for the fact its own canonical procedure named, and
would have failed the phase on every attempt. It stayed invisible for a whole round because
the PURE path inlines that same file as context: the contract was true, the delivery was not,
and both paths' tests were green. Rule: when you change what a contract TELLS a leaf to read,
count how that thing reaches EACH transport that receives the contract — the must-read set, the
inlined context, the template's own words — and name the one for each. The tell is a corrected
sentence that names a file, in a commit with no diff to the code that assembles what the leaf is
handed. Same question as the paragraph above, asked from the other end: there the transport
moved and the contract did not, here the contract moved and the transport did not. - Surface 13 — a gate that refuses a SPELLING does not refuse a POINTER to where that spelling
lives. Every guard in this repository that protects a leaf-read surface decides by reading the
text for a token: the severity allowlist refuses a severity VALUE, the restatement scans refuse a
known sentence. Afile:linecitation contains none of those tokens and passes, while telling
the reader exactly where to find them — so the guarded fact reaches the leaf anyway, by
reference. Issue #168: adocs/RUNBOOK.mdbullet was rewritten to name the constants behind a
cost claim, with fourtools/*.py:<line>citations and the sentence "Read the four constants for
the run you are looking at".docs/RUNBOOK.mdsits in the CLOSED judgment-rule list of both
verifySKILLs, whose closing sentence forbids readingtools/for judgment rules, and the
cited module holdsclassify_verify_severity— which says that indevamajorfail-closes
the run while aminorbuys a warm same-phase repair turn. A leaf that follows the instruction
learns which grade is the cheap exit: the hand-assignment the allowlist exists to prevent,
delivered by a pointer the allowlist cannot see. Rule: when you add a REFERENCE to a surface
some check guards, ask what the TARGET contains that the check forbids HERE — the check cannot
ask it for you. Prefer naming the fact, or the canonical document that owns it, over naming the
file and the line. The tell is a diff that adds no token any guard looks for, which is why
no gate, no mutation sweep and no census sees it; what found it was a CORPUS DELTA on the
citation FORM —os.walkfortools/*.py:<line>over the leaf-read documents, zero before the
branch and four after, and the guarded document was the only one in the tree carrying any. Take
that count on every surface a check guards, not just the file you edited. Distinct from surface 5,
where caller-controlled data pollutes a classification the host computes: nothing here is
caller-controlled, and the guard is simply answering "does this text CONTAIN the thing" when the
question is "does this text DELIVER the thing".
2. Confirm the path production actually takes
Production does not necessarily pass the argument you wrote the rule on. In PR #51 the rule
went on run_syntax_check's sources argument, and the workflow never passes sources (the
auto-discovery side went straight through).
- Read the conductor's in-process calls for real (
_build_inproc/_gate_lint_check/_gate_syntax_check/_execute_inproc) - Confirm the position of the check too. Placed after a skip or an early return, it is inert
under that condition - Check whether two places read the same fact, via
references/dual-read-pairs.md - If what you are adding is a conductor in-process (deterministic) substep, work
references/deterministic-substep-wiring.mdbefore writing anything — it lists the sites that
special-case one, acrosstools/workflow_conductor.pyandtools/orchestration_runtime.py, and
every miss recorded there left the unit suite green while the real flow failed closed
There are two kinds of position. Look only at control flow inside the function and you miss
one.
- Inside the function: is it after the skip / early return (above)
- Inside the pipeline: does it run before the side that reads the value? PR #57 put the
normalization in the "right function", but the caller had already rendered the prompt from
the un-normalized value 15 lines earlier: the symptom did not go away and it newly created an
audit-trail mismatch — worse than before the fix - There is one way to find this. Do not poke the validator in a unit test; drive the production
entry point end to end and assert on the final product (the rendered prompt, the persisted
JSON) - A SYNTHETIC input measures nothing for a value the production code resolves from somewhere
else. Before building a fixture around an input, find out where the code under test actually
reads that value from. Two instances on PR #104, both of which produced a test that passed while
observing nothing: a witness drove the Bash guard with a syntheticrepo_rootin the payload,
but~+is$PWDandtools/hooks/common.pyexpands it fromos.getcwd(), so the anchor
pointed at wherever pytest was started; and a spy's frame walk anchored onos.getcwd()was
handed probe modules written into atempfiledirectory, so it missed on every self-test and
fell back silently. The fix in both cases was to move the process, or to make the root an
argument — not to make the fixture more elaborate - A DISPOSITION TOWARD BAD INPUT travels when you copy or parametrise a path, and the receiving
side's validator may not accept it. Copying "a missing file degrades to""" from a sibling
path is the recorded shape: on issue #168 the new path's every value was a DECLARED key, and the
declared-key validator refuses an empty one — insiderecord_launch, from a call the loop did
not guard — so the loop's own named fail-closed branch was bypassed and the exception aborted
the conductor instead. A refusal that happens one frame too late converts a recoverable named
outcome into a crash, which is strictly worse than either the degradation or the refusal.
Whenever you lift a behaviour into a shared loop or copy it to a second caller, ask what the
RECEIVING side does with the degraded value, not what the donor did.- And the test for it has to cross the same frame. The row that "pinned" that degradation
drove the builder and then called the validator BY HAND, so it saw a cleanValueErrorwhere
production saw aRuntimeErrorescaping the substep. Same class as the synthetic-input bullet
above, one level up: a fixture can be right about every value and still stop short of the
frame where the failure happens. Drive the production entry point far enough that the recovery
branch is what you observe.
- And the test for it has to cross the same frame. The row that "pinned" that degradation
3. Decide the failure's attribution
Every refusal you add gets routed to someone as their fault. The criteria and the known branches
are in references/failure-routing.md. The essentials: what the leaf can fix is a content
failure; what the conductor, the IR, or the environment caused is a transport fail_closed. Name
exceptions by type; do not catch at the width of except ValueError (that burns the leaf's retry
budget on the conductor's own defects).
Once attribution is decided, decide when it should surface. Attribution can be right while the
moment is wrong. Do not fail mid-run on something detectable at launch — the precedent isREQUIRED_CLI_TOOLS in tools/run_workflow.py. Order of decisions: (a) whose fault is it →
(b) where is the earliest place it can be detected reliably; if (b) is earlier than the gate,
add the earlier check and keep the gate's fail-closed as a backstop. (Eight subagents across
four rounds never raised this; Codex raised it on its first pass — dependencies, preflight and
operations are surfaces subagents structurally do not look at.)
The cause a refusal message names must match the actual set of causes. For a fail-closed,
failing correctly is not enough: if the party that failed is a leaf, it is closed only once the
message is an instruction under which a warm retry converges (L174 handed back an instruction
that could not converge in principle).
- When the causes grow, grow the message. Do not assert "the measured cause is X" in the singular
- For parsers, say that the reported position drifts; do not let it read as "the cause is here"
- If the enumeration does not close, say so and describe the shape to look for
- Order the causes by REACHABILITY, most reachable first — this applies to any remedy a check
prints, an operator's included, and it is surface 6's question asked of the message instead of
the command. Issue #71's roster check named a vendor cause when the shortest path was a one-linepermissions.deny, and following the printed remedy subtracts the tool from the required set
permanently: the check goes green by having been WIDENED rather than satisfied - A remedy must not be followable by HALF. A leaf doing only the first half of "re-issue it
against a granted directory, or drop the leading/and pass the directory aspath" is ALLOWED
by the hook (rc=0, measured) and the tool then matches nothing. Silent empty is the worst
answer a boundary can give — indistinguishable from a true negative, so the leaf reports
absence and stops. When a remedy is a conjunction, say so, and say what doing one half produces - A refusal that carries ANOTHER PROGRAM's message keeps its HEAD, and one reason is not all
the reasons. A tool states why it refused near its start and ends with generic advice; a[-N:]cut keeps the advice and drops the diagnosis, and a fixture with a one-line fake
message cannot see the difference (atmofab-review-loop's threshold rule: the fixture must
straddle the cut). And a refusal that blames one cause for every failure of a step (a failed
launch reported as "the two programs are from different installations") sends the reader to
fix what is already right: split the branches by what the observation can actually tell
apart. Issue #316 shipped the second, and then the first inside the fix for it;references/failure-routing.md§"A refusal that quotes another program" - A remedy that names
--rederive <phase>is a claim about the keys DOWNSTREAM of that
phase, and a re-derived phase that reproduces its output moves none of them. A downstream
key holds the upstream's OUTPUT HASH, so a rebuild whose bytes come out identical leaves the
downstream certified output eligible, and it is adopted again — the step the remedy was meant
to re-run silently does not run. Issue #333 PR-2 told the operator to follow a re-mapped
execution site with--rederive build, "which re-derives Validate"; a reviewer rebuilt a real
source tree at two paths and got identical binaries, and the old site's Validate was paired
with the new site's binary (the branch's own acceptance run had shown the skip and nobody read
it as the defect). Rule: name EVERY phase the remedy must re-run (--rederive build,validate), and run the remedy on a byte-identical reproduction before writing it.
The same holds for any prose saying "X re-derives Y": it is true only when X's output changes.references/failure-routing.md§"A remedy that re-derives one phase"
4. Tests pin properties
- Do not transcribe measured values. A test asserting "
language: nullresults in dual
ownership" became a pin that rejected the correct fix the moment the implementation moved
to structural reading. If you use a measured value, compute it from the code inside the test - Pin at the handler, not the helper. Deleting the caller's wiring and staying green
actually happened (4 of 5 sites) - Keep one test that pushes a production payload through the real validator. The conductor's
tests mock each tool function, so they never traverse the validation layer - A mock that replaces a function WHOLESALE hides every change to its signature. When a
change adds a parameter to a function, everymock.patch(<fn>, return_value=…)(or a lambda
with the old arity) of it stays green while a real caller that was not updated raisesTypeError— and a caller insidetry: … except Exceptionturns that into a silent wrong
answer. Issue #284 PR-2 shipped two such callers through a full green suite; a billed--with-depsrun found them. Rule: when you change a signature, (a) make every mock of that
functionautospec=True(or a lambda with the new arity) and assert the NEW argument in at
least one row, and (b) check every non-test caller binds to the new signature — an AST pass
that binds each call's argument shape toinspect.signaturedoes it in one command
(references/judgment-episodes.md§"a whole-function mock hid a missing argument" carries it).
And assert the value, not just the presence: a row checking the new argument equals the
DEFAULT value is killed by nothing that hardcodes the default — run it under a non-default
value (the same PR's rounds 1 and 2 found six rows that could not tell a hardcoded default from the real value).
The same holds when the change makes an INPUT required rather than adding a parameter, and
(b) then covers TEST callers behind a host skip too: a row skipped for a missing tool is a
caller the green suite never ran. Issue #424 PR-2 madederive_build_graphrefuse a toolchain
naming no build system, rewrote the 39 call sites intest_codegen_bundle.pythe suite ran,
and left the one CUDA build-through row that skips without the compiler driver — red with the
driver on PATH, found in round 2 by Codex and the security reviewer independently. Enumerate call sites withgit grep, not with the suite - Do not call a sample a pin. A test placed outside the place that defines the set cannot
claim set identity — it can only sample rejections. Write in the docstring what is pinned and
what is sampled, and if the predicate has several branches (==/startswith/ trailing
slash), each branch needs its own probe — PR #55 said "pinned" three rounds running and was
broken three different ways, the third being "a file and not a directory", a branch nobody
had probed - If one string states two rules, a substring pin is necessarily true via the other one. This
repo's remedy, hint and contract texts fold two rules into one message, and PR #76 hit this four
times on one branch. Remedy: split on the half the rule governs, then read
(WRITE_HINT.split("For temp files")[1]). Decision procedure: before writing a pin, read the
whole message and count whether the same word is used in explaining another rule — one use and
the substring pin does not hold. Confirmation runs to "the mutation makes it fail": run it
againstorigin/main's wording and see it actually fail. Episode:references/judgment-episodes.md§Appendix
5. Run the mutation check before committing
Revert your fix one hunk at a time and confirm the tests fail. In PR #51 the survivors among
42 mutants, and the tests passing for the wrong reason, were things I learned only when a later
round pointed them out.
The procedure and the script are owned by atmofab-review-loop
(.claude/skills/atmofab-review-loop/scripts/mutation_check.py).
The same thing runs before review, so read its "Before you hand it over (round 0)".
When the rule rests on what a vendor TOOL can reach, mutation says nothing — measure the
tool. scripts/measure_claude_tool.py in this skill drives a real Claude Code tool
through a loopback stand-in, unbilled, in a saturated fixture, and exits non-zero when any
row fails its declared expectation. Reach for it before writing "this spelling cannot
reach outside" — that claim was written down wrong four times on issue #71, each time from
a probe that measured something else. references/verification.md carries the details.
Its own tests live beside it and pytest tools/tests does not collect them:python3 -m pytest .claude/skills/atmofab-enforcement-change/scripts/tests -q -p no:randomly.
6. Verify and record
Run the procedures in references/verification.md (suite baseline, ruff diff against
origin/main, doc size ceilings, end-to-end through a real server process, the prose grep) and
write the measured values into the commit message or the pull request. Do not write an assertion you have not
measured.
7. Decide whether this skill itself needs updating
At the end of the work, judge whether this session made this skill's content stale, or exposed
a gap it should fill. Do not stop at the judgment: when you decide something is needed, tell
the user (do not rewrite silently — what changes and why is material for their decision).
Signs to look for:
- You created a new dual-read pair, or resolved an existing one → the table in
references/dual-read-pairs.md - You hesitated over failure attribution, or it fit no existing category →
references/failure-routing.md - A verification step was missing, or a command had gone stale →
references/verification.md - You added or changed a conductor in-process (deterministic) substep →
references/deterministic-substep-wiring.md - A new input surface, or a new episode of an existing one →
references/input-surfaces.md - The mutation check gave a false positive, or missed something →
.claude/skills/atmofab-review-loop/scripts/mutation_check.py - The skill did not fire when it should have, or fired when it should not →
description - You broke one of the three judgment rules while following the procedure → SKILL.md itself.
Distinguish "the rule was missing" from "the rule was there but its trigger point was
elsewhere" - A trap not written here consumed your time → a candidate for addition. But check that it
is not a copy of a canonical source (rule content belongs to the repo's docs; only the
procedure and traps written nowhere else belong here), and put the EPISODE in the reference
file and the RULE here — that split is what keeps this file loadable - Memory (
feedback_enforcement_change_skill.md) is a pointer only. Wanting to add content
there means the skill is what needs updating
If you judge that no update is needed, say so in one line. The judgment itself is information.
When triaging review findings
Use this skill even when you have not touched code. Judgment rule 1 is what matters at this
moment.
- Ask what the leaf GAINS before reaching for a reproduction (rule 1-e). The findings that
survive the most rounds here die on that question, and attacking first buys evidence the
classification never reads - Before sorting a finding into real / false positive / residual, run the reproduction
- Treat "the implementation is right but the test is weak" as real (PR #51's surviving mutants
came from here) - When a finding lands inside the previous round's fix, that is not a coincidence. Look
hardest at the files you just fixed in the next round