Vulnerability research

The load-bearing rule

A finding you cannot reproduce is not a finding. It is a hypothesis, and it stays one until a command you can name turns it into evidence. Report the hypothesis as a hypothesis or drop it — never as a vulnerability.

This is the rule the whole activity hangs off, because the economics run against you. Generating a plausible vulnerability narrative is cheap and verifying one is expensive, so an unverified report moves your cost onto whoever reads it. The curl project watched its confirmed-vulnerability rate fall from over 15% to under 5% as unverified AI reports arrived, and closed a seven-year bug bounty on 31 January 2026 rather than keep paying that tax. The recovery is the instructive half: by April 2026 the slop had stopped and the AI-assisted reports that replaced it were mostly valid, because reporters started verifying before sending.

Scope this skill holds

This skill fires when the deliverable is a claim about real software being exploitable. That is a different job from the reviews this tree already has, so pick the right one and do not run two:

• A pending diff before a PR → peer-review.
• A single-pass look at the current branch → the built-in security-review.
• A whole-codebase multi-axis audit ending in a fix plan → parallel-audit.
• Proving an isolation layer still engages, with tests → stress-test.
• A vulnerability claim you intend to act on or send somewhere → this skill.

Deliver the finding set you were asked for. Do not widen a bounded question ("is this one path exploitable?") into a full audit.

Hunting with /goal

Use /goal for an open-ended hunt on one named surface. A bounded question gets no goal. In Codex, set the goal only when the user asks for one. After each turn a small model judges the condition from the transcript alone. So write the condition as the evidence the judge must SEE, never as "a vulnerability was found".

• State the outcome, not the method. Ask for exactly one finding that breaks a quoted claim of the target's threat model (this tree's is SECURITY.md), at evidence level 1 of §2. Name a method only when you want that method ("use Hypothesis").
• List what does not count: SECURITY.md's out-of-scope list and § Known limitations, a finding a §3 tool already reports, evidence below level 1, and any edit to an existing test, hook, lint, deny rule or allowlist entry.
• Make "nothing yet" not met, except at the cap. The judge reads only the condition, so the cap is a second met-clause: the transcript shows the turn count and §7's clean-run report.
• Run an unattended goal inside a glovebox, never on the host.
• Aim at what is ASSUMED, not at what is proved. TLC already checks the invariants under docs/tla/. Hunt in the constants a config hand-sets to stand for shipped code, in a clause a model assumes (Consent.tla's MintedLabels = FALSE), and in the unchecked A- rows of docs/merge-queue-fsm.md §7.

A goal that clears gives you a CANDIDATE. §1 through §6 still apply to it.

/goal Find exactly one request that the egress verdict service admits on a host the
allowlist tags `ro`, where the request writes. Met when the transcript shows a new test
under tests/ that fails on this tree, its failing pytest output, and the quoted SECURITY.md
§ Defense layers 2 sentence it breaks. Does not count: a host that stores what a GET says
(SECURITY.md already lists it); a code trace with no failing run; any edit to an existing
test, rule or allowlist entry. Also met when the transcript shows 40 turns and a clean-run
report naming every request shape tried and what each returned.

1. Read the threat model before you read the code

Score every candidate against `SECURITY.md`, not against a generic vulnerability pattern. This tree states its threat model explicitly: the agent itself is the attacker, and it assumes that agent can run any shell as glovebox-agent, read its own guardrails, and spawn sub-agents. It also states what is out of scope — outside attackers, host compromise, base-image supply chain, deliberate bypass flags, and post-session merged code.

A candidate that lands in the out-of-scope list is not a finding here, however real it is elsewhere. Say so in one line and move on. A candidate that restates something already in SECURITY.md § Known limitations is not a finding either — that file already documents the deny rules as speed bumps, the LLM monitor as defense-in-depth rather than a boundary, and the audit log as tamper-evident rather than tamper-proof. Claiming any of those three as a discovery is the most common way to waste a reviewer's afternoon.

The question that earns a finding: does this break a claim SECURITY.md actually makes? Name the claim, quote it, then show the break.

2. Reproduce it, with a command that fails

Build the smallest artifact that demonstrates the break, and state the command that runs it. What counts, in descending order of strength:

1. A test that goes red on the current tree and green once the hole is closed. This is the standard writing-tests sets for non-vacuity, and it applies here unchanged. Follow that skill for the mechanics.
2. A transcript of the real boundary refusing or failing to refuse — a live vm egress probe, an audit-log entry, a hook verdict. .github/workflows/vm-live-checks.yaml is where this tree runs that class against real KVM.
3. A worked trace through the code with every branch quoted. Weakest, and admissible only when you say plainly that you could not execute it.

Say which of the three you have. "Ran X, saw Y" and "expect Z because the code reads this way" are different epistemic states, and root CLAUDE.md § Supervision-legible work requires you to keep them apart. A finding at level 3 that you present at level 1 is the failure this skill exists to prevent.

When the boundary cannot run where you are — no /dev/kvm, no vm binary — do not silently drop to level 3 and call it confirmed. Dispatch the live check that has the hardware, per the `ci-triage` skill, and say what you dispatched.

3. Corroborate with the tools already wired in

Do not re-derive by hand what a tool in this tree already answers. The winning AIxCC systems were hybrids — Team Atlanta took first place in August 2025 with classical fuzzers and symbolic execution feeding language-specific AI assistants, and the competition's own lesson was that when one method faltered another caught the bug. Reasoning alone is the weakest leg of that tripod.

What is already here, and what each answers:

Question	Tool	Where
Does this pattern appear elsewhere in the tree?	bandit, plus ast-grep over this repo's own ruleset	.github/scripts/sast-scan.sh, glovebox-monitor/src/monitorlib/astgrep-rules.yml
Is this a known CVE in a dependency?	osv-scanner	.github/scripts/deps-vuln/scan.sh (osv-scanner.toml is the accepted-advisory list, currently empty)
Is this a known CVE in the guest image?	grype	.github/scripts/grype-scan.sh (.grype.yaml is the ignore list)
Does a data-flow path reach this sink?	CodeQL	.github/workflows/codeql.yaml
Would a test have caught this?	Stryker, the bash mutation lane	.github/workflows/mutation-testing.yaml
Does the parser survive hostile input?	Hypothesis over the untrusted-input parses and the landing simulation	.github/workflows/fuzz.yaml
Does the design permit this sequence?	TLC over the TLA+ models, and the guard-conjunct mutant sweep	.github/scripts/checks/tla-model-check.py, .github/scripts/tla-guard-mutants.py

A finding that one of these already reports is not yours to re-file — check first. A finding that contradicts one of them is interesting, and the contradiction goes in the report.

When no tool answers your question, build the one that does. Keep it as a test that fails before the fix and passes after: that red-to-green pair is level 1, and a harness that only records behavior is level 3.

The monitor's own write-scan covers less than its audit entry suggests, so never read a clean static_scan record as coverage. It runs only on the write tools (risk.is_write_tool, read from config/write-tools.json), so code written through bash — a heredoc, tee, sed -i — is never scanned, in any mode. A write no language scanner reads, such as a .md file or a notebook cell, records a skip note rather than findings. It records only high- and medium-severity findings, so findings: 0 means none at those tiers, not none seen. Run .github/scripts/sast-scan.sh over the tree when you need the ruleset's real reach.

4. Refute before you escalate

Before a finding leaves this session, spawn one general-purpose sub-agent whose brief is to refute it. Give it the reproduction and the threat-model claim, and tell it to default to "refuted" when the evidence is ambiguous. One agent, not a panel — reach for three only when the finding would trigger a public advisory.

This is not a re-read of your own reasoning, which root CLAUDE.md and `prompt-authoring.md` both tell you to skip. It is a fresh reader who never saw you build the case, which is the independence peer-review buys. Cite the model tier from root CLAUDE.md § Delegation: an adversarial review is Opus, high.

Know what the refuter cannot do for you. It shares your training and your failure modes, so it catches an invented detail, a broken chain of reasoning, and a line you misread. It does not catch a misconception you both hold about how this system works. Two models agreeing is therefore weaker evidence than two people agreeing, and it never upgrades a level-3 trace into a level-1 reproduction. The reproduction is what substitutes for expertise the reader may not have — a command that goes red is checkable by someone who cannot audit your argument, and this project's maintainer has said plainly that they cannot.

Record the refuter's verdict in the report even when it agrees with you, and record it as one model's opinion.

The second judge is a different model family: the Codex review on a PR in this repo that carries the red test and its fix. config/review-severities.json makes its P0 and P1 findings block the merge. Record its verdict beside the refuter's. A real boundary bypass goes to private reporting (SECURITY.md) and opens no PR, so a human decides whether to open one. Otherwise write that the second judge is missing, and say which case applies: no PR reviewed yet, a finding outside this repo, a private report, a Codex-originated session, a declined review, or SKIP_CODEX_REVIEW set.

5. Bound what you send outside this repo

A report to an upstream maintainer costs their time, not yours. OpenSSF and OpenJS both converged on the same reporter-side rules in 2025 and 2026, and they are cheap to follow:

• Send a private proof of vulnerability, and a proposed patch where you can write one.
• Give reproduction steps that run, and impact stated against that project's threat model.
• Never send a finding you have not reproduced, and never send a speculative pattern match.

Never open a public issue, advisory, CVE request, or upstream security report yourself. SECURITY.md routes findings through a private advisory, and the human decides what goes outside. Big Sleep works this way too — the agent flags, a human researcher verifies, and only then does the bug move. Write the report, put it in the PR body or in chat, and say it is ready to send.

This is the one place in this tree where "ask first" beats the autonomy default, and root CLAUDE.md § Autonomy already carves it out: disclosure is outward-facing and irreversible.

6. Write the report

300 words maximum per finding. Longer is not more rigorous; it is less read.

Each finding carries exactly these, in this order:

1. The claim it breaks — quoted from SECURITY.md, with its heading.
2. The reproduction — the command, and which of the three evidence levels it reached.
3. What an attacker gets — in one sentence, against this threat model.
4. The refuter's verdict, and Codex's.
5. The fix, or the reason you did not write one.

Nothing else. No severity theatre, no CVSS vector you derived by feel, no restatement of what the subsystem does. If a detail would not change what the reader does next, cut it.

A finding that reads right:

Breaks: SECURITY.md § Defense layers 1 — "the removal of an old host-wide policy file leaves no trace of glovebox behind."
Reproduction: level 1. A dangling parked symlink under managed-settings.d/ makes Path.is_file() read False, so bin/lib/uninstall_settings.py's _teardown_settings_file reports it removed without deleting the link.
Impact: the stale link keeps shadowing the path Claude Code reads, so a plain claude still runs a hook pointing at a checkout glovebox uninstall deleted.
Refuter: confirmed — reproduced independently against a fresh dangling symlink. Codex: missing — no PR reviewed yet.
Fix: check is_symlink() and not exists() before the is_file() guard, and add the regression test above.

And one that does not, for contrast — this is the shape to delete:

The credential handling in the monitor appears potentially insecure. An attacker might be able to access sensitive material under certain conditions, which could have serious security implications. Recommend reviewing the permission model. Severity: HIGH.

It names no claim, runs no command, and gives the reader nothing to check.

7. When you find nothing

Say that, and say what you covered. A clean result with a named surface is a real deliverable — it tells the next session where not to look again. Numbers over adjectives: "checked whether a resume can be pointed at another workspace's transcript — the archive is workspace-keyed at both capture and restore, so it cannot" beats "the resume path looks fine."

Never pad a clean run with a speculative finding to make it look productive. That is the slop failure with extra steps.