How to watch a dispatched grok worker, converse with it across checkpoints, and route work to the right model. Three observation planes and what each can answer - the unified log is the only one that records CAUSES. Model capability table with verified capabilities and measured failure modes. Read when a worker looks stalled, before choosing a model, or before resuming a thread.
Install
npx skillscat add simnova/openclinxr/grok-worker-monitoring Install via the SkillsCat registry.
Watching a worker, talking to it, and routing to the right model
Complements: orchestrator-dispatch-loop (how to dispatch/integrate), delegated-worker-contract
(the worker's obligations), agent-session-continuity (resume/session-id rules),large-task-orchestration (fan-out). This skill owns observation, conversation and routing.
1. THREE OBSERVATION PLANES - and only one records causes
Most wasted diagnosis comes from reading the wrong plane. They answer different questions.
| plane | path | answers | does NOT answer |
|---|---|---|---|
| ledger | .openclinxr/openclaw/worker-sessions.jsonl |
did it spawn? did it complete? turns, proofsOk, model, worktree | anything mid-flight |
| transcript | ~/.grok/sessions/<url-encoded-cwd>/<sessionId>/updates.jsonl |
WHAT it is doing - tool calls, command titles, growth rate | WHY it stopped |
| unified log | ~/.grok/logs/unified.jsonl |
WHY - inference retries, empty responses, auth-lock contention, startup timing | fine-grained tool detail |
The unified log is the one people forget. It is documented in the bundled guide
(~/.grok/docs/user-guide/, "Internal log files") and it is where a stall explains itself.
L=~/.grok/logs/unified.jsonl
grep "<sessionId-prefix>" "$L" | grep -iE "retry|error|fail|429|rate" | tail -6The stall signature that cost a full dispatch
A worker at 0.1% CPU, transcript frozen, zero files, no crash. Read as "hung on a bad brief".
The log said otherwise:
{"msg":"shell.turn.inference_retry",
"ctx":{"attempt":1,"max_retries":15,"kind":"empty_response",
"reason":"empty response from model (no_visible_content)"}}It was a provider failure in retry backoff, not a briefing failure. Every symptom is explained:
low CPU = waiting on backoff; no transcript growth = nothing to emit; no crash = still retrying.
Rule: symptoms constrain a cause, they never name it. Before attributing a stall to a brief,
grep the unified log. It is one command.
Liveness checklist, in order
# 1. Is it alive and burning CPU, or idle?
ps -o %cpu=,etime= -p "$PID"
# 2. Is it emitting? (compare over a real window, not once)
L1=$(wc -l < "$D/updates.jsonl"); sleep 120; L2=$(wc -l < "$D/updates.jsonl")
# 3. Is it isolated where you think? (NOT the main checkout)
lsof -a -p "$PID" -d cwd
# 4. WHY did it stop?
grep "<sid>" ~/.grok/logs/unified.jsonl | grep -iE "retry|error" | tail
# 5. Is there work to preserve before any kill?
git -C "$WORKTREE" status --porcelain | wc -lMonitoring traps, each paid for
pgrep -f <pattern>matches the monitor's own command line - the pattern is in it. Reported a
finished worker as live for a full cycle. Grep a durable artifact instead.pkill -f <tool>kills your own worker - it is running that tool too. Kill by PID, and guard
with|| true. Never start a delegate while a long local job is running.- Never set
RUST_LOGorGROK_LOG_FILEfor debugging - grok logs the bearer API token in
plaintext. Read the existing unified log instead. - Do not judge a worker's TREE mid-flight - it produced a false accusation once. Reading the log
to check for a stall is fine; grading artifacts is not. - A worktree that looks empty may hold a commit.
git status --porcelainis clean after the
worker commits. Checkgit log --oneline main..HEADtoo. - Diff against the MERGE-BASE, not
main. Commits landed on main after the worktree was cut show
as deletions inmain..HEADand look like the worker deleted your files.
2. MODEL ROUTING - verified capabilities and measured failure modes
Update this table when a capability is measured, not when it is claimed.
| model | route | context | tools | vision | status / failure mode |
|---|---|---|---|---|---|
ox-alpha (OpenRouter stealth/ox-alpha) |
[model.ox-alpha] in USER config, env_key = OPENROUTER_API_KEY |
1,048,576 | VERIFIED - wrote and read a file in 3 calls | VERIFIED - see calibration below | Free promotional window. RISK: returns empty_response / no_visible_content on large accumulated context (observed ~325 turns deep) and the harness then retries up to 15x, looking exactly like a hang. |
deepseek-v4-flash / deepseek-v4-pro |
[subagents.models] USER config |
1M | yes | NO - text only | Cheap tier. Returned HTTP 402 Insufficient Balance on every dispatch in the 2026-08-21 session. Check balance before routing here. |
grok-4.5 |
spawn-time model |
- | yes | yes | The sanctioned fallback on a 402 only. |
grok-4.6 |
superagent thread, --resume <id> |
- | yes | yes | Never for worker tasks. Reserve for: a grade it owes, a measurement that contradicts it, a worker UNABLE:, a write-scope collision. |
Config facts that bind
- Project
.grok/config.tomldoes NOT merge[subagents.models]. Child routing must go in the
USER~/.grok/config.toml. Proven 2026-08-04. - Never put a literal
sk-...in any.toml. Useenv_key = "NAME"and export in~/.zshrc,
outside the repo. Verify withgrep -rl "sk-or-v1-" .returning nothing. - Spawn-time
modeloverrides config. A role whose policy names a higher tier needsmodelDowngradeReasonor the dispatch fails closed (#461). - A model must be listed by
grok modelsbefore--model <alias>resolves.
Vision calibration - what "multimodal" was actually verified to mean
Do not adopt a vision claim; test it against images whose ground truth you hold, using a filename that
gives nothing away, and offer an explicit "CANNOT VIEW" escape.
| probe | ground truth | result |
|---|---|---|
| texture atlas under a neutral filename | leopard print + rose panels | described rosettes on golden-mustard, the trim band, and the five-block lower third - added detail the grader had not recorded |
| 60x28 native crop at 6x LANCZOS, unwarned | SMOOTH | SMOOTH |
| same crop with a real 3px zigzag injected, unwarned | RAGGED | RAGGED |
| 100x120 thumbnail, fine pose question | not pixel-resolvable | rejected interpolated upscale on its own, switched to nearest-neighbor + pixel classification |
The pair matters, not the single positive. A model that always answers SMOOTH passes the second
row. The injected-zigzag control is what proves it discriminates.
What vision does NOT change: the producer/grader split. A worker may grade ANOTHER slice's
artifacts. It may not be the sole grader of its own output - a fabricated score.json in this
repo's history was an agent scoring itself, and seeing does not make a model disinterested. Put this
in the brief verbatim: your visual report on your own output is EVIDENCE, not a verdict.
3. CONVERSATION - checkpoint returns, not fire-and-forget
A live worker is not addressable. The session id is in the ledger the moment it spawns, but that
is for kill-recovery; two processes writing one updates.jsonl is a real hazard. Talk at returns.
Staged dispatch
Split done_when into groups. Do not invent stage: rules - briefFromIssue refuses narrative
proofs. Stage N's gate is the existing rules for that group; integrate still requires the full list.
The brief MUST say this, verbatim, or staging never happens:
Returning with GROUP A green and GROUP B UNRUN IS SUCCESS for this process.
Without it the worker finishes the whole slice - the same gradient that makes "optional wiring" never
happen.
Checkpoints are ARTIFACTS, not pauses
Measured failure: a "giant effort" brief said do not stop at a checkpoint, and the worker read that
as no artifacts until done. It performed 82 correct sequential actions and wrote zero files,
because:
- the first write was ill-defined - the brief coupled the artifact's schema to its last consumer,
so nothing could be written until the whole chain was understood; - there was no early
exists:proof, so intermediate files had no contractual existence -
reading was safe, writing was risky, and the gradient was all-reconnaissance; - "GIANT" scaled preparation to the declared size.
Fix, and it is wording:
STAGE GATE:
<a -> b -> c -> d -> e>. Proof 1 of 5 is due in your first ~10 actions: write<first artifact>before ANY downstream code. Its schema is pinned below - do not read<consumer>
first. Checkpoints are artifacts, not pauses - never stop the process, never skip a stage
artifact.
Resuming
OPENCLINXR_WORKER=1 GROK_SUBAGENTS=1 OPENCLINXR_RAW_GROK_SANCTIONED=1 ~/.grok/bin/grok -p "<short delta>" --resume <sessionId> --model <same> --output-format json --max-turns <n> --cwd <worktree>- NEVER
dispatch({worktree:true, resume})- it resets the worktree and destroys the work. - Never resume a slice whose ledger shows
spawnedwith nocompleted- it is still live. - The three env vars are not optional: without
OPENCLINXR_WORKER=1a SessionStart hook runs doc
hygiene and dirties the tree before the worker acts. - A wrong-but-existing session id confabulates rather than erroring. Take it from the ledger
programmatically; confirmSession ... found locallyon stdout.
What to put in a delta
Hand back numbers and what you accept, never a story.
- the measurement, in full, including how you obtained it
- what you ACCEPT of its work, by name, so it does not redo it
- an explicit non-accusation when you disagree with a judgement rather than a fact
- the ONE measurement that splits the remaining ambiguity
Rank methods if you have a preference; never rank causes - a ranked cause costs a turn of
story-fitting. "Candidates, unranked, and they may all be wrong" is free.
4. BRIEF HYGIENE THAT SHOWS UP IN MONITORING
- Any number in a brief becomes a target. Telling a worker "the source has 66 joints" makes a
worker whose clip drives 31 pad to 66. Say: that number is context; report the measured count. - Convert
it.fails(toit(in the same change that makes a clause pass. Arun:proof on a
planted RED passes only while the defect stands - contract-green can mean nothing was fixed. ghbodies with backticks go through--body-file, never an inline double-quoted string -
the shell expands them and mangles the record. Same for commit messages (-F file).- Give a stopping condition, not a work list. Parallel delegation makes over-building nearly free.
4b. GIVE ABSOLUTE PATHS FOR EVERY OUT-OF-REPO INPUT
An early-artifact instruction cannot bind if the worker cannot reach the input.
Measured. A brief said "proof 1 of 5 is due in your first ~10 actions" and named its source ashuman-base-animations.glb - a bare filename. The file lives OUTSIDE the repo under~/.openclinxr-tools/.... The worker burned roughly ten actions searching for it:
find . -iname "*human-base-anim*"
grep -rln "human-base-animations" --include=...
mdfind -name "human-base-animations.glb"
find /Users/patrick -...
D=/Users/patrick/.openclinxr-tools/mesh2motion-app; find "$D" -name ...By the time it had the path, the "first ~10 actions" budget was gone - the instruction did not fail,
it was made unreachable. The worker did nothing wrong; searching was the only way to proceed.
Rule. Every input the brief names gets an ABSOLUTE path if it is outside the repo, and a
repo-relative path if inside. Verify each one resolves before dispatching:
for f in "$SRC_CLIP" "$TARGET_MAP" "$SUBJECT_GLB"; do
[ -e "$f" ] && echo "ok $f" || echo "MISSING $f" # a MISSING here is a brief defect, not a worker one
doneThis is the §6k family - name the probe that already works. Naming a thing is not naming where it
is. The same applies to tools installed outside the repo (Blender addons, provider caches, model
clones): give the full path once, in the brief.
Corollary for monitoring. When a worker's action trace shows repeated find / mdfind /grep -rln for the same token, that is not exploration - it is a missing path in your brief. It is
visible in the transcript within the first few minutes, and it is cheap to prevent and expensive to
watch.
5. QUICK REFERENCE
# who is live, and where
ps -eo pid,etime,command | grep '[.]grok/bin/grok'
lsof -a -p "$PID" -d cwd
# the three planes
grep '"slice":"issue-N"' .openclinxr/openclaw/worker-sessions.jsonl # spawned/completed
D=$(find ~/.grok/sessions -maxdepth 2 -type d -name "<sessionId>"); wc -l < "$D/updates.jsonl"
grep "<sid>" ~/.grok/logs/unified.jsonl | grep -iE "retry|error" | tail # CAUSES
# distinct actions (is it looping, or traversing?)
python3 -c "import json,re;[...]" # dedup consecutive update titles
# safe kill (worktree checked FIRST)
git -C "$WT" status --porcelain | wc -l && git -C "$WT" log --oneline main..HEAD
kill "$PID" 2>/dev/null || true