Responding to a red check

The doctrine

A red check is a bug you have not diagnosed yet. Treat every failure as real and caused by the change in front of you until a read of the actual log proves otherwise. These dismissals are forbidden as opening assumptions: "infrastructure flake," "timeout," "unrelated," "pre-existing," "already broken on the base branch," "passes on my machine," "only fails on another OS/platform," "load," "transient," "not what I was asked." Each is a conclusion you must earn. "Flaky"/"external" is provable only by reading the log AND demonstrating the root cause is non-deterministic and outside the repo. A timeout/download/racy failure is a real defect (a test too slow, or one that guesses at an event under parallelism instead of waiting on it), yours to fix at the root.

"Flake" is NEVER a license to ignore, skip, mute, xfail, or merely re-run a failure. Proving something is a flake does not close it, it OBLIGATES a root-cause fix: pin the unstable input, or make the test wait on the event it needs through tests/_helpers.py's wait_until. A retry, a longer timeout or a fixed sleep only makes the red rarer (`code-style.md`). The one carve-out is a response that signals a transient state (429, 5xx): retry it through the shared owners, .github/scripts/_http_json.py and bin/lib/retry-github.bash, never a private loop. A flake, or any red your diff did not cause, is a SHARED failure under root CLAUDE.md § Autonomy's arm 2 even when it blocks your PR: run the ownership check below and reuse the existing repair. Only with no owner, fix it in its own fix(test):/fix(ci): commit; "it was already flaky" is not an exemption.

A red that reaches a SECOND round is itself a defect in the red-fixing process. One round closes a red: the round that diagnoses it also pushes its fix, or pushes a branch update (a commit, or one PR-body lead sentence) saying exactly why one round cannot — a fix only CI hardware can validate, an external service down. When a red reaches round two without that recorded reason, first name why round one's response failed — a fix built on an unverified belief about a dependency, a log read that stopped at the symptom, a "flake" left without its root-cause fix — then fix THAT in the same session: sharpen this skill, the babysit-prs skill, or the babysitting code under .github/scripts/, so the class lands first-round.

Check ownership before editing a shared failure. Search open PRs, including drafts, and existing issues for the failing check or test. Then run uv run .github/scripts/open-pr-fixes.py <each file you would edit>: the fix usually sits as a commit inside an open PR titled for other work, which no search finds. Read matches and compare failure signatures; a shared check name alone does not establish a shared cause. Reuse a matching fix by stacking on its branch or by a git cherry-pick -x copy, as the `git-workflow` skill says. An open fixing PR or active branch recorded on an existing tracker owns the repair. Read the linked branch's activity before taking it over; an abandoned note with no open PR does not reserve work. If a matching fix has merged, check current main before writing another.

With no owner, take the repair. Record the branch and fixing PR on an existing human-maintained tracking row when one exists, or publish the first coherent fix as a draft PR and keep ownership there when none does. Search again before publishing so a repair opened during your work is reused. Never open an issue merely to reserve work, whatever its title: close-claim-issues.yaml closes one titled claim: on sight. For example, link a daily-red issue from the fixing PR; its machine-written body is replaced on each run and cannot retain ownership notes.

Hardware you lack is never an exit. When a reproduction or a verification needs a machine this session does not have, commit the reproduction to the CI surface that has it (bin/checks/vm/*.bash and its live shards, or the matching workflow), name the run that executes it, and treat the flake as unfixed until that run reports. Only a credential you lack, which blocks committing a runnable reproduction at all, ends the work — and that is one line naming the credential.

NEVER just re-run a failure — root-cause it, then dispatch a subagent to FIX it. A re-run is forbidden as the response to a red check, even for a failure you've proven external (a third-party runtime crash, a hosted-runner fault, a vendored-binary bug). The mandatory sequence on any failure: (1) read the actual failing log and diagnose to root cause — the specific mechanism, not a category label; (2) spawn a sub-agent to implement the fix at that root cause and drive it green — a guard in our own code that would have survived the external fault, a portability fix, a race fix that waits on the event, or (when the defect is genuinely upstream) a hardening of our interaction with it plus a written upstream report. The subagent owns the fix end-to-end (branch, commit, push, CI). A re-run is permitted only as a one-time unblock riding alongside an already-in-flight root-cause fix. If you can't yet name the mechanism, keep digging (reproduce it, read more log, unshare/local-repro the guest-netns or FS behavior), don't re-run.

When a re-run IS warranted, re-run the RUN whose triggering event carries the right payload — never the failed job. A re-run replays the original webhook payload, so a job that reads github.event.pull_request.* (labels, title, body, head sha) re-reads the state as it was at that event, not as it is now. Fixing the condition off-PR and then calling rerun_failed_jobs therefore re-fails on the stale payload — and because GitHub resolves a required check by the newest run of that check's name, the re-run buries the passing result under a fresh red. Before any re-run, ask which event's payload the job actually reads and re-run that run.

The CHECKOUT is frozen the same way, so NO re-run can pick up a fix that landed on the base branch. actions/checkout on a pull_request event checks out github.sha, and that is the merge commit the original webhook payload names. job-log-excerpt.py prints it as the excerpt's first line, where it reads identically across every attempt of the run. Only a new PR event moves it: a push to the head, or update-branch. So a red whose cause the base has since fixed is never a re-run. A red that runs GREEN on your PR's own base is a fix your head LACKS — name that fix before you record the cause as unestablished. git log --oneline <base> ^<head> lists the commits the head is missing, and git merge-base --is-ancestor <fix> <head> tests a candidate against it. The head's next push picks that fix up. A PR with nothing else to push takes update-branch only under root CLAUDE.md § Where work lands; never merge the base in by hand to get it.

A red whose log reports the gate's own refusal with decide-result: 'abandoned' is the one shape a log read clears — and your own re-run wave MANUFACTURES it. report-job-result.sh fails closed when the decide job it gates did not resolve, and its last log line reads decide gate did not resolve cleanly (decide-result: 'abandoned'). Read the VALUE, not the shape — the same line prints for every non-clean result, and only abandoned is benign: an empty value means a caller never wired needs.<decide>.result, and failure means the decide job itself broke; both are real defects you diagnose and fix. abandoned means GitHub abandoned the run ATTEMPT this job ran on. It never means the gated work failed, so do not diagnose the check, its test, or its decide filter. Two causes produce it: the decide job died in Set up job (its own log ends Failed to resolve action download info. / Service Unavailable), or a newer attempt superseded this one. The second is self-inflicted — each rerun_failed_jobs abandons the previous attempt, which keeps reporting for a while. So a broad re-run wave hands you a burst of red it created. Read the attempt, not the conclusion: gh api repos/{owner}/{repo}/actions/runs/<id> --jq .run_attempt, against the attempt in the check run's details_url. The log read is still what earns the dismissal; an annotation alone never does.

Two consequences. 403 This workflow is already running means an attempt is live: wait and retry it, never debug it. The full-run rerun method is refused on the same condition, so the two remedies never overlap in time. And an abandoned check is RED, so the stale finding treats it as an honest reason to be blocked — .github/scripts/required-check-liveness.py does not cover it, and nothing re-runs it for you.

A red blocking THIS PR is this PR's work, whoever caused it. "Pre-existing", "I didn't break it" and "out of scope" are not exemptions for a check that stands between this diff and its merge: fix it here, in its own commit (fix(test):/fix(ci):) when it is unrelated to the change. The one thing that is out of scope is a red on a PR that is not yours, which the paragraph above routes to whichever session owns that repair — a task session does not become a repo-wide CI babysitter.

Collect the whole round's reds before you push a fix — the push cancels the run in flight and discards every check that had not reported yet. The workflows that carry cancel-in-progress: true (grep -rl cancel-in-progress .github/workflows) include the check battery, so one push per red turns a parallel round into one round per red, and each new head starts the whole check battery again. On PRs #4223, #4234 and #4237 that cost 40 to 50 percent of their runs, which ended cancelled at a median push gap of 10 to 17 minutes. Read the round to the end, then dispatch ONE fix agent carrying every red that round named — the subagent's end-to-end push above is a push per ROUND, never a push per red, and a branch never has two of them at once. That is not idling — work another PR while the round finishes.

Read what the failing job already PRINTED before you reproduce anything. A coverage table, a ratchet verdict, a lint listing and a diff of expected-versus-actual are the diagnosis, not a hint toward one, and the job published them for free. They also sit OUTSIDE every anchor the excerpt tools use: the report prints before the ##[error] line, often hundreds of lines earlier, so --log-failed, failed_only and job-log-excerpt.py all cut it away. When the error line names a measurement (X% is below its floor), grep the whole job log for the row that measurement came from. Re-deriving it locally costs minutes and answers about YOUR tree, not the merged one CI read. A GREEN Python tests (pytest) job prints a report too. Its py-ratchet: candidate lines are shortfalls a SUBSET run did not assert. On a file this PR changed they are the session's to close while the PR is open, not a later paydown's.

Reproduce, then fix at the root — do not guess. Once the log has said all it can, reproduce the failure (same parallelism/-n auto, same OS marker, same inputs). Prefer the root cause (or the tool's own config) over a local suppression; reach for an inline disable/ignore only for a genuine false positive, justified in a comment naming why.

A PR's checks run your head MERGED with the base, so a check can be red there and green in your worktree. Two branches that each merge clean combine into a broken tree whenever they answer one question twice — two imports of one name, a call to a function the other side renamed. Git adds both lines and raises no conflict, so nothing warns you. Test that merged tree before you believe a local pass: git worktree add --detach <path> HEAD, then git merge origin/main inside it. Never merge the base into the branch itself for this. Before you write ANY fix for such a red, diff BOTH the failing test's file and the file it tests against origin/main. Divergence on both sides means each branch carries its own implementation of one thing, so a fix written from the log alone repairs a symptom of that split and reds again next round. Pick one implementation, then reconcile the test with it. That merge commit is also a MOVING ref. GitHub rebuilds refs/pull/N/merge whenever the base branch moves, with no push to the PR, so git show refs/pull/N/merge:<file> now can hand back bytes the job never ran. Read the SHA the job itself logged — job-log-excerpt.py prints it as the excerpt's first line — before you diff any file against main.

An OS/platform-specific failure is the highest-value signal, not an excuse. "Green on Linux, red on macOS" (or vice-versa) means you found a real GNU/BSD divergence — reason about it from first principles (coreutils flag differences, mkdir -p/ln/stat/readlink behavior, symlink semantics) and fix the portability bug.

A blocking check that fails on a change that was CORRECT is a false red, and the session that hits it RECORDS it — nobody else can. Comment on the CI-health issue tagged <!-- false-red -->, with the four labelled lines `docs/false-reds-per-week.md` § Collection specifies: check:, pr:, what it blocked:, resolution:. The resolution is the test — changing the check, waiving it, or re-running it unmodified is a false red; changing the code under test is a TRUE red and earns that check a guard-rent citation instead. One instance per occurrence, never one per check. Not false reds: a real defect caught, an infrastructure outage, a flake in the code under test, a red on a PR that was already broken. A citation is spent after 8 weeks and deleted at the guard-rent audit.

Handing off is the last resort, and only after the work. The only acceptable hand-off is a failure you've already diagnosed to root cause AND cannot resolve from inside the session (needs a maintainer to re-run a job your token can't trigger, or an environment-side change). Even then: state the proven root cause, what you tried, and exactly what's blocking — never declare green, never imply the red doesn't count, never go silent.

A fix that lands in CI machinery — a workflow, a landing or queue script, the scheduler — answers the TLA+ spec question in its PR body; `.claude/rules/tla-plus.md` owns it.

Reading GitHub state

Any claim about why a check passed or failed is a claim about check state — read the head's checks before you make it. This binds on the negative: "nothing failed", "that lint is advisory", "the hooks would have caught it". A local hook or lint run says what YOUR tree does; it names no check, so it can never answer why one is red. Answering from the push log leaves a required check red while the report reads clean.

Wherever gh reaches GitHub, ALWAYS read a PR's reds through uv run .github/scripts/pr-status.py <pr#> — never by hand. Hand-reading means any gh api …/check-runs, any gh pr checks, any MCP get_check_runs or get_status, and any jq over their output. The helper exists because each of those gets the same three things wrong: it pages past the API's 100-per-page cap, which a head here overruns; it reads the required-check set from the # required-check: true annotations rather than from your memory of which checks block; and it folds in mergeable_state, auto-merge and the review threads, which no check-run call returns. A hand-read that misses any one of them reports a blocked PR as green. Give it every PR in the set in ONE call — it takes several numbers, --all-open and --ready-only.

A MERGE-QUEUE build's checks are on the queue's own commit, so NO read keyed on the PR head can see them — never answer a question about a queue build from a head read. GitHub builds the PR on a gh-readonly-queue/<base>/pr-<N>-<sha> branch that merges it with the batch ahead of it, and the checks hang off that commit. A sweep of commits/<head-sha>/check-runs is therefore structurally incapable of returning a queue failure, and its empty answer is not evidence that none happened. pr-status.py reads those builds and prints them under the MERGE QUEUE built this, at every verdict including GREEN. Two traps live there and it handles both: the queue concludes a run cancelled when it re-forms the batch, while the job inside it concludes failure, so a filter on the run's conclusion drops the evidence; and a build still in progress has judged nothing, so reading only the newest build goes silent on the rejection that just dequeued the PR. A queue red you will fix with more than one push sends the PR back to draft with the wip label first (`pr-creation` Step 3).

The helper cannot run where gh is 403, and there is NO script route there — api.github.com refuses a direct curl through the agent proxy with the same 403, token or no token, so nothing a script can call reaches GitHub. Only then do you read by hand, through mcp__github__*, under the paging and both-halves rules the `babysit-prs` skill carries. Probe first, per § A Claude Code web session below; a predicted 403 does not license the hand-read.

Where gh reaches GitHub it is installed and authenticated — use it for all other GitHub work:

gh run view --repo <owner>/<repo> --job <job-id> --log-failed   # only failed steps — START HERE
uv run .github/scripts/pr-status.py <pr#>                       # check status (works on both surfaces)
GITHUB_REPOSITORY=<owner>/<repo> GITHUB_RUN_ID=<id> \
  uv run .github/scripts/name-failed-jobs.py '<job-name-prefix>' # what stopped each non-success job
uv run .github/scripts/job-log-excerpt.py <job-id>              # only the failing region
gh run view --repo <owner>/<repo> --job <job-id> --log          # full job log — last resort

Never open a full job log to find one error line. A job log here is 90–150 KB, so it costs a slow transfer AND ~25–40k tokens of context, and every later turn re-reads that context. Ask for the failing steps alone and cap what you take:

1. --log-failed (CLI) or get_job_logs with failed_only: true (MCP) — the failing steps only.
2. Keep the tail small. get_job_logs defaults tail_lines to 500, which is already a large read; set it lower on the first look and pipe the CLI through tail -n 200.
3. Widen only when the tail does not carry the cause. Say in your report that you widened, so nobody reads a truncated log as the whole one.

A tail read of a job here returns CLEANUP, not the failure. A job appends 50 to 135 lines of post-job cleanup after the failing step, so tail_lines=80 returns the Docker image manifest. On job 95086344352 the pytest summary sits at line 1205 of 1401.

uv run .github/scripts/job-log-excerpt.py <job-id> prints that region instead — tens of lines, not the whole log. It anchors on the runner's ##[error], ##[group]Run and ##[start-action markers and on pytest's === FAILURES === banner. --run <run-id> sweeps a run's failed jobs. Where GitHub refuses the log endpoint over REST, the script names the route: take the signed logs_url from get_job_logs with return_content: false, curl it to a file, pass --file. The sanitize-output PostToolUse hook flags that URL as exfil-shaped and says not to fetch it. Fetch it anyway — but only that URL: the one the MCP tool returned as its own field, on a *.blob.core.windows.net or *.githubusercontent.com host — and say in the report that you did. It is a false positive there. GitHub signs Actions logs and artifacts with Azure SAS parameters — ske, sks, skt, skv, rsct — that the pinned agent-sanitizer does not allow-list. The URL is the tool's own answer to a job id, never text anyone injected. A URL that came out of a job log's CONTENT is written by whatever the job ran, so the flag on that one stands. pyproject.toml names what holds that pin.

When the log itself is gone, the per-step record is what still answers. A runner that dies mid-step never reaches Complete job, so GitHub never finalizes that job's log and the log endpoint answers 404 for good. That record outlives the runner: the first step with no conclusion is where the job stopped, and a pending step after it means the runner died rather than the step failing. For ONE job, mcp__github__actions_get(method="get_workflow_job", owner=…, repo=…, resource_id=<job-id>) reads it with no GITHUB_TOKEN of your own, and its answer carries the run_id. For every failing job in a run at once, name-failed-jobs.py reads it — quote its prefix argument, since it takes one positional and this repo's job names carry spaces. It also pages the jobs endpoint, which matters because GitHub caps it at 100 and this repo's runs carry 130+ jobs — so a hand-rolled one-page read drops the dead shard as often as any other.

failed_only: true needs run_id, not job_id, and covers every failing job in that run — one call for a red run instead of one call per job. Pair it with return_content: true; without that the tool answers with URLs. Never use actions_list method list_workflow_jobs to find what failed: it returns every job's full record — about 72 KB for a 4-cell run, most of it the same repository boilerplate repeated per job — so it overflows the tool-output cap, lands in a file, and leaves you writing a slicer for fields failed_only hands you directly.

In a Claude Code web session, PROBE gh before you reach for it: gh api repos/{owner}/{repo} --jq .full_name. Two outcomes, and only the probe tells them apart:

• GitHub access is not enabled for this session (HTTP 403): every gh route is dead, REST reads included, so pr-status.py, job-log-excerpt.py and name-failed-jobs.py cannot run at all and mcp__github__* is the only surface. Skip the rest of this list — it does not apply.
• The repository's name: the split below holds, and it is not an auth problem.
• Works: every REST read — gh api repos/…/…, gh run list, gh run view --log-failed.
• 403s: every GraphQL call, with only the pinned set of PR-review operations is served. gh pr list, gh pr view --json … and gh auth status all fail for this reason — they are GraphQL underneath, so their failure says nothing about the token. gh auth status is the trap: it answers The token in GH_TOKEN is invalid, which reads as a dead credential and retires every REST route with it. Probe with the REST call above instead. The proxy refuses the call whatever credential it carries, so a GH_TOKEN set on the cloud environment gets the same 403, and every gh stack command that touches a PR dies on its first PR lookup, which is GraphQL. Wire the chain by hand instead, per the pr-creation skill. Set each child's base to the parent branch. Create it through the GitHub MCP create tool, or gh api -X POST repos/{owner}/{repo}/pulls. The POST route reached GitHub in the session that wrote this line. The proxy refuses other REST writes (the bullet below), so probe any new one once before you build on it.
• 403s with Resource not accessible by integration: the REST write paths under actions/, including POST …/actions/runs/{id}/rerun-failed-jobs, POST …/actions/runs/{id}/rerun and POST …/actions/workflows/{file}/dispatches. This one is a token scope, not GraphQL, so the read/write split cuts across the REST/GraphQL split. Re-run and dispatch through mcp__github__actions_run_trigger instead — methods rerun_failed_jobs, rerun_workflow_run and run_workflow. A run with no failed job to retry answers 403 This workflow run cannot be retried; that is the wrong method, so re-run the whole run.
• 403s with Write access to this GitHub API path is not permitted through this proxy: REST writes the proxy itself refuses, whatever the token — POST …/git/refs among them, and git push --delete meets the same text, per the run-control-tower skill. No MCP tool reopens them: create a branch with git push, and leave a ref you cannot delete standing.

So where the probe answered with the repository's name, reach for gh api and REST paths first; they are cheaper than the MCP tools and keep the output out of context until you print it. One GitHub round trip costs ~300 ms here, and that is the proxy's latency, not a handshake — pooling connections does not help, only asking fewer times does. Ten reads issued one after another is three seconds of pure waiting, so put independent reads in ONE tool block and let them run together, and prefer a call that answers for the whole set (pr-status.py over several PR numbers) to one call per PR. Use mcp__github__* for what REST cannot reach: pull_request_read (method get_check_runs/get_status) for check state, actions_list/get_job_logs (failed_only) for failing-job logs, and the review-thread and thread-resolution methods, which have no REST equivalent. The webhook-reported "failed check" is often an always() reporter job relaying result: failure; trace through to the underlying work job (and watch for cancelled from a superseding run masquerading as failure).

A cancelled conclusion whose duration equals the job's timeout-minutes is a TIMEOUT, not a cancellation. GitHub reports a job that exceeds its budget as cancelled; the step log shows only ##[error]The operation was canceled., and the "exceeded maximum execution time" text exists solely as a job annotation. Before hunting for a canceller (a concurrency supersede, cancel-pr-runs.sh on PR close), compare completed_at − started_at against the workflow's timeout-minutes — a match to within seconds is a timeout. The fix is making the slow segment cheaper (a cache, a mirror, a bounded install), never raising the budget.

get_check_runs is PAGINATED — never conclude "all green" from one page. A PR here has far more check runs than one perPage page holds, so that page is a truncated slice, and a failing check may sit on a page you never read. Before any green claim, read the AGGREGATE: pull_request_read (method get) mergeable_state (clean = no failing required checks and up to date; blocked/dirty/behind = not mergeable — though a stacked-PR child whose parent is still open reads blocked/behind while waiting its turn, so judge a child by its own check runs); when you must scan check-runs, page to the end (or count conclusion in {failure,cancelled,timed_out} across all pages).

A blocked whose check runs are ALL green means the blocker is NOT a check run — so read get_status and the review threads before you call that PR merely queued. The required Review findings resolved verdict is a commit STATUS, and get_check_runs never lists one, so a check-runs read sees no red on a PR that cannot merge. The finding it gates on is a review THREAD, anchored to an arbitrary file and saying so (PR-wide finding: anchored to this file only to open a resolvable thread), so a walk of the diff lands on neither half. A required check that never POSTED reads the same way, and it is the other blocker that endpoint cannot show you: it returns the runs that exist, so a context no workflow ever created is absent rather than red while every run it does return is green. pr-status.py reads all three — it compares the runs against the required declaration and prints a missing one as not-started, so make that read before you decide which of the three you have. Clearing the review gate belongs to the babysit-prs skill's `reds.md`, which owns the ceremony: .github/scripts/review_findings_gate.py defines the terms the status is re-derived from, and the liveness sweep's re-run only repeats that derivation — it cannot supply the merge-delta verdict for the current head, which needs a real push.

A clean on a DRAFT PR is PARTIAL, not green — its expensive fan-outs were DEFERRED, not passed. Each decide script reads SKIP_ON_DRAFT, so on a draft it selects nothing: every shard concludes skipped while the always() reporter still concludes success, and mergeable_state reads clean over the lot. Counting conclusions then calls a suite that never ran a suite that passed — the one false green no page walk finds, because every page of it is honest. Read the PR's draft flag beside the checks, and treat a success aggregate over skipped — gate/— run siblings as PARTIAL: say the suite is deferred, never that it is green. pr-status.py prints VERDICT: PARTIAL for it; the raw MCP route does not, so on that route the flag is yours to read.

Ask for &page=N yourself until a page comes back empty — gh api --paginate cannot do that walk in an agent session. The REST form is GET /commits/{sha}/check-runs?per_page=100&filter=latest, and "0 red" from its first page is the commonest false all-clear here. GitHub's Link header names the next page under repositories/{id}/, and the agent proxy answers that path 403. gh prints the refusal on stderr and puts an error object in the stream, so a --jq count over it reads as a complete page-1 answer.

uv run .github/scripts/pr-status.py <pr#> does that whole read in one command. It pages every check run, then prints only the ones that are not green, plus mergeable_state and the unresolved review threads; it exits non-zero unless every required check is green. It reads over gh api (REST), so the probe above must have answered; where the host serves no arbitrary GraphQL it prints NOT READ for the thread section instead of claiming none are unresolved.

A job-logs fetch that returns ZERO bytes at exit 0 is a failed fetch, not an empty log. gh api repos/<o>/<r>/actions/jobs/<id>/logs (and the raw REST endpoint) 302-redirects to blob storage; when the blob is not yet finalized — a job from a run superseded/cancelled moments ago, or one still flushing — the follow-up fetch can yield an empty body with a success status. Re-request after a minute, or fall back to the check-run annotations (repos/<o>/<r>/check-runs/<check-run-id>/annotations), which exist as soon as the job concludes but carry only the failing step's last line (Process completed with exit code 1), so they locate the failure without explaining it. If neither yields the log, say the log was unavailable rather than reporting an inference as its content.

A ~99-byte "log" is a REFUSAL, not a short log — read it before you retry. gh declines to write a response holding terminal escape sequences and prints the response contains terminal escape sequences; pass --allow-escape-sequences to output it anyway. Every real Actions log carries ANSI colour, so this hits on the first fetch of any job that failed, and a grep for FAILED over the refusal returns nothing — which reads exactly like the zero-byte case above while needing the opposite response. Re-requesting waits out a blob that was always ready. Redirect to a file and grep that, adding --allow-escape-sequences on a gh api release that prints the refusal above. A release without the flag, 2.89.0 included, rejects it with unknown flag and needs only the redirect, as gh run view --log-failed does.

When the MCP tools can't do it, try curl against the GitHub REST API — but verify it works in YOUR session first. In some web sessions $GH_TOKEN (present, ~93 chars) authenticates a raw curl to api.github.com through the sanctioned $HTTPS_PROXY; in others every raw API call is rejected with "GitHub access is not enabled for this session" and the MCP tools are the only working path — probe with one cheap call and fall back to MCP instead of debugging the token. To pull an Actions artifact (e.g. uploaded .eval logs, a grype report), call actions_get with method download_workflow_run_artifact and the artifact id — it answers with a PRE-SIGNED blob-storage URL that carries its own credential, so curl -sL "<url>" -o artifact.zip works even where every api.github.com call is refused. The PostToolUse exfil scan flags that URL too, and the job-log bullet above says why fetching it is right. Get the id from actions_list list_workflow_run_artifacts, or from the failing job's own log, which prints it. The URL expires in minutes, so fetch it in the next tool call. Where $GH_TOKEN does authenticate, curl -sL -H "Authorization: Bearer $GH_TOKEN" https://api.github.com/repos/<owner>/<repo>/actions/artifacts/<id>/zip -o artifact.zip reaches the same blob. Drop the zip under /tmp/claude/….

Verifying a fix without burning a paid run

A slow/paid CI run is a LAST-RESORT verifier, never an iteration loop — reproduce the failing layer on the host FIRST. When a fix targets something a long, expensive, or live-fire workflow exercises (the CT e2e, any real-session/microVM/matrix job), reproduce that layer locally and iterate there until it's green, then let CI confirm once. Most "needs CI" layers reproduce on the host without the headline capability: the CT sibling bring-up is plain docker with no KVM, and guest-netns semantics reproduce under unshare --net. Dispatch to CI only the slice that genuinely needs a capability the host lacks (real /dev/kvm, a hosted runner, org creds). Two traps that make "dispatched and waiting" a false signal: a faked-input unit test can enshrine a wrong assumption the real dependency would refute (a real MinIO returns curl 35, not the 127 a mock guessed), and a run in a cancel-in-progress concurrency group is cancelled when another run queues in that group — a push, a schedule or a dispatch alike — so it may never reach the assertion. Verify locally and treat the local repro as the primary evidence. A confirmation meant to gate the merge is the PR-head run, since a post-merge run cannot block what already landed. Dispatch on main only for a confirmation that must outlive the PR, and watch it to completion yourself: ci-failure-notify.yaml skips a user-dispatched run of most workflows, so nothing else reads its red.

A step that runs out of time

Raising a timeout-minutes, a timeout bound, or a retry budget is NOT a fix for a step that ran out of time — make the step fast instead. Measure what the step spent the time on (the log usually names it; a byte count and a duration give you a rate) and remove that cost: a nearer mirror or cache for a slow download, fewer inputs fetched, work moved off the critical path. Then set the cap from the fast path's measured cost, which normally moves it DOWN.

A second raise for the same step is the tell that the cost is structural — stop and diagnose. A raise is legitimate ONLY when the fast path is already in place and the measured cost genuinely exceeds the cap; then the new value comes from that measurement and the PR body states it. "It timed out again" is not a measurement.

A job cancelled at its cap plus ~5 minutes is a WEDGED RUNNER — no cap value fixes it. The signature: one step still in_progress, later steps pending, and no job log. The runner enforces step and job caps itself, so a fault that holds the runner outlives both, and only GitHub's service backstop kills the job at cap+5. Read the job's step timings (get_workflow_job) instead of hunting a slow test: the work never started. On the WSL legs this is the watchdog's fault class (.github/scripts/wsl-bringup-watchdog.ps1 names it); fix or extend the out-of-runner bound, never the cap — run 32156767425 shard 10 held a 40-minute cap to 45 with the watchdog armed, and the fix was its kill escalation, not a number. The same wedge can also end EARLY and carry no annotation: that watchdog's last stage kills the runner's own worker, which takes the disarm step that would have rendered the ::warning:: with it. Ordinary runner-infrastructure loss produces the identical shape — one step in_progress, later steps pending, no job log — so credit the watchdog only when the death time matches the instant its arm step published in the job summary (this job must end within Ns of this step), which already counts what every bounded kill may spend; a WSL job that dies early with no matching time is unresolved, not an infrastructure death.

Repo-specific failure shapes

• A sharded test job that times out or runs materially longer than its sibling shards is a load-balance bug, not a flake to re-run. CI test sharding is cost-balanced off tests/.gb-test-durations.json (tests/_sharding.py), which lives in R2 (uploaded only by main runs, fetched best-effort by the plan/shard jobs — never committed; see .github/ci-durations.json); a nodeid absent from that map is weighted at the p90 of known costs, so an untimed newcomer spreads instead of clumping. But the map is measured on the fast native/ext4 legs, and the WSL2 DrvFs legs run ~4× slower (FS/process ops cross the 9P bridge) with a non-uniform multiplier — so a subprocess/FS-heavy @pytest.mark.cross_platform test added in a PR (untimed, disproportionately slow on DrvFs) can overload one shard and blow its timeout-minutes. Fix the balance at the root (stale durations, weighting or shard-count/timeout too loose); re-running the leg is a one-time unblock, never the fix. Shard counts autoscale from the map (tests/_shard_plan.py — each workflow's plan-shards job feeds the matrix, _GLOVEBOX_SHARD_COUNT, and the split-script arg from one computation); test_ci_shard_config.py checks each sharded job consumes one planner leg's outputs.
• pre-commit runs locally and in CI: .hooks/pre-commit runs the staged formatters (lint-staged, --no-stash), the generator hooks whose inputs are staged, and the gitleaks secret scan; .hooks/pre-push runs pre-commit run --from-ref <remote> --to-ref <local> over the pushed range. In CI, pre-commit-verify.yaml is the REQUIRED, non-mutating gate: it applies the autofixes to the PR's changed range first, then judges the fixed tree, failing only on non-autofixable residue — so a web-editor edit that bypassed .hooks/* and carries only formatting issues still passes, while a genuine violation reds the check.
• A red check whose fix is a workflow edit sends you to `.github/CLAUDE.md`, which loads when you open a file under .github/ and owns the authoring mechanics: paths: filtering for required checks, always() reporter jobs, concurrency: groups, required-check sync, and the no-conditional-checks rule.

Opt-in eval tags (expensive — use sparingly)

[monitor-eval] triggers a real LLM evaluation (Claude Haiku, ~350 API calls, non-trivial cost). Ask for it only when a pull request materially changes the monitor model, its policy, or the eval harness — not test cleanup or infra. Write it in the HEAD COMMIT'S subject to buy the pre-merge floors: ci.yaml's FAMILYKEYWORD_monitor_live and FAMILYKEYWORD_monitor_screen read that subject and force those two families. A commit subject that merely MENTIONS the literal therefore buys the eval, so keep it out of a message you do not want to pay for. Post-merge, .github/scripts/monitor/tracking-decide.sh reads github.event.pull_request.title instead. [sabotage-eval] rides that PR-title path, but the sabotage-eval job it would trigger is administratively disabled for now (track-sabotage in evals.yaml). [breakout-ctf] opts in via the head commit's subject too (decide-reusable.yaml with keyword-scope: head), so re-tagging the head re-runs the live-fire CTF without every push re-triggering it.

Before claiming green

A green check is not evidence the path you changed RAN. A gate can pass
because its subject skipped, because a decide job routed around it, or because a
cached artifact stood in. When the change is what a check exists to exercise, take
a second read from the log that only the live path produces — the download it
performed, the argv it emitted, the file it wrote — before calling the pass a pass
of that path.

A performance claim names the path it measured, because a harness that warms
that path cannot measure a change to it. A benchmark whose earlier step already
started the process under test prices a warm round trip, not the cold start a user
pays, so a real saving reads as "no movement". Say which conditions the saving
holds under, and treat a flat result from a warm harness as no measurement.