Pan Oversee — Overdeck Framework Testing

Purpose

This skill exists to test the Overdeck framework itself. The issue being overseen is a test payload — the real goal is exercising the full agent lifecycle pipeline and identifying every bug, glitch, and rough edge along the way.

Supervise an agent working on an issue through the entire lifecycle:
spawn/resume → work → completion → review → feedback loop → test → merge-ready

This is NOT passive monitoring. You are actively:

• Watching for bugs at every stage — dashboard rendering, agent spawning, workspace creation, xBRIEF tasks lifecycle, specialist handoffs, status transitions, terminal output, API responses
• Documenting everything — no bug is too small. If something flickers, logs a warning, shows stale data, or doesn't match expected state, that's a finding
• Fixing infrastructure bugs when they block progress, then continuing the test
• Filing or updating GitHub issues (PAN-XXX) for every bug found

Bug Tracking During Oversight

Maintain a running bug log throughout the session. For each finding:

1. Note it immediately — don't wait until the end. Capture the symptom, what you expected, and what actually happened.
2. Check for existing issues — gh issue list --search "keyword" before filing duplicates
3. File new issues or update existing ones — include reproduction context from this oversight run
4. Categorize severity:
   • Blocker — stops the pipeline, had to fix inline to continue
   • Bug — incorrect behavior but pipeline continued
   • Cosmetic — dashboard display issue, misleading status, stale data
   • Observation — not clearly a bug but worth investigating

At the end of the oversight session, report a summary to the user of all findings: what was filed, what was updated, what was fixed inline, and what needs follow-up.

Usage

/pan-oversee PAN-129

The argument is an issue ID (e.g. PAN-129). The skill handles the rest.

Phase Detection (CRITICAL — Always Run First)

Before doing anything, detect what phase the issue is currently in. Do NOT assume the issue is at the beginning. Run all checks and jump to the correct phase.

ISSUE_ID="PAN-{ID}"
ISSUE_LOWER=$(echo "$ISSUE_ID" | tr '[:upper:]' '[:lower:]')

echo "=== Phase Detection for $ISSUE_ID ==="

# 1. Dashboard running?
DASHBOARD=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:3011/api/health 2>/dev/null)
echo "Dashboard: $DASHBOARD"

# 2. Workspace exists?
WS_PATH=""
for dir in ~/.overdeck/workspaces/feature-$ISSUE_LOWER ~/Projects/*/workspaces/feature-$ISSUE_LOWER; do
  if [ -d "$dir" ]; then WS_PATH="$dir"; break; fi
done
echo "Workspace: ${WS_PATH:-NONE}"

# 3. Agent state?
AGENT_STATE=$(cat ~/.overdeck/agents/agent-$ISSUE_LOWER/state.json 2>/dev/null)
echo "Agent state: ${AGENT_STATE:-NONE}"

# 4. Live pane? /api/agents reads the host's terminal backend (Herdr by
# default, tmux only under terminal.backend: tmux); never `tmux -L overdeck`,
# which sees no Herdr agent.
LIVE_PANE=$(curl -s http://localhost:3011/api/agents 2>/dev/null \
  | jq -r --arg id "agent-$ISSUE_LOWER" '.[] | select(.id == $id) | "\(.id) pane=\(.paneState) live=\(.hasLiveTmuxSession)"')
echo "Live pane: ${LIVE_PANE:-NONE}"

# 5. Completion marker?
COMPLETED=$(ls ~/.overdeck/agents/agent-$ISSUE_LOWER/completed 2>/dev/null)
echo "Completed: ${COMPLETED:-NO}"

# 6. Where is it? PAN-3917: there is no stored review/test/merge row — the
# derived state is computed from the tracker, the branch, and the pull request.
REVIEW_STATE=$(pan show "$ISSUE_ID" --json 2>/dev/null)
echo "Derived state: $REVIEW_STATE"
(cd "$WS_PATH" && gh pr view --json state,reviewDecision,mergeable,statusCheckRollup) 2>/dev/null || true

# 6b. Branch-portable xBRIEF pipeline mirror (corroborating, not authoritative)
PIPELINE_MIRROR=""
if [ -n "$WS_PATH" ] && [ -f "$WS_PATH/.pan/spec.vbrief.json" ]; then
  PIPELINE_MIRROR=$(jq -c '.plan.metadata.pipeline // empty' "$WS_PATH/.pan/spec.vbrief.json" 2>/dev/null)
fi
echo "xBRIEF pipeline mirror: ${PIPELINE_MIRROR:-NONE}"

# 7. Specialist activity?
SPECIALISTS=$(curl -s http://localhost:3011/api/specialists 2>/dev/null)
echo "Specialists: $SPECIALISTS"

Phase Decision Matrix

Based on the checks above, determine the current phase. The forge is
authoritative: the PR's reviewDecision, its checks, and whether it merged are
the review/test/merge state (PAN-3917 — there is no status row to read). Use the
branch-portable xBRIEF mirror only as a corroborating signal; if it disagrees
with the forge, log a bug and prefer the forge.

Condition	Current Phase	Jump To
No workspace, no agent state	Not started	Phase 0 → Phase 1
Workspace exists, agent active + live pane	Agent working	Phase 2
Workspace exists, agent active, no live pane	Agent crashed/stuck	Phase 2 (recovery)
Workspace exists, agent stopped, no completion	Agent gave up or crashed	Phase 1 (resume)
PR open, reviewDecision is REVIEW_REQUIRED or null	Awaiting review	Phase 4
PR reviewDecision is CHANGES_REQUESTED	Feedback loop	Phase 5
PR approved, checks still pending	Awaiting tests	Phase 6
PR approved, checks green, forge says mergeable	Merge ready	Phase 7
PR approved, checks red	Test failed	Phase 5 (test feedback)
PR merged	Done	Report success

Print which phase you're entering and why, e.g.:

"Issue PAN-129 is in Phase 4 (awaiting review) — the PR's reviewDecision is REVIEW_REQUIRED, skipping to monitor review agent."

Supervision Workflow

Phase 0: Pre-flight Check

Only needed if dashboard isn't running or Cloister isn't active.

# Dashboard running?
curl -s http://localhost:3011/api/health | jq .

# Cloister running? (needed for specialist handoffs)
curl -s http://localhost:3011/api/cloister/status | jq '{running, lastCheck}'

# Specialists initialized?
curl -s http://localhost:3011/api/specialists | jq '.[] | {name, state, isRunning}'

If workspace doesn't exist, create it:

pan workspace create PAN-{ID}

If Cloister isn't running, start it:

curl -s -X POST http://localhost:3011/api/cloister/start

Phase 1: Spawn or Resume Agent

Check if agent exists and resume or spawn:

# If agent state exists and has a session ID — resume
pan resume PAN-{ID}

# If no agent state — spawn fresh
curl -s -X POST http://localhost:3011/api/agents \
  -H 'Content-Type: application/json' \
  -d '{"issueId": "PAN-{ID}"}'

Or use the dashboard UI via Playwright:

1. Navigate to https://overdeck.localhost
2. Find the issue card for PAN-{ID}
3. Click "Start Agent" or "Resume"

Phase 2: Monitor Work Phase

Poll the agent's activity every 30-60 seconds. Watch for:

Signs of progress:

• The agent's output shows tool calls, file edits, test runs
• lastActivity timestamp in state.json is updating
• Heartbeat file is fresh: ~/.overdeck/heartbeats/agent-pan-{ID}.json

Signs of trouble:

• No output change for > 5 minutes → agent may be stuck
• Error messages in the agent's output
• Agent asking questions with no one to answer
• Auth errors (wrong API key, provider issues)

Monitoring commands:

# Watch live output (non-blocking; reads the pane on Herdr or tmux)
curl -s "http://localhost:3011/api/agents/agent-pan-{ID}/output?lines=30" | jq -r .output

# Check heartbeat freshness
cat ~/.overdeck/heartbeats/agent-pan-{ID}.json 2>/dev/null | jq '{timestamp, tool_name, current_task}'

# Check activity log
curl -s http://localhost:3011/api/agents/agent-pan-{ID}/activity | jq '.[-3:]'

If stuck: Poke the agent or send a message:

pan tell PAN-{ID} "Are you stuck? Please continue working on the task."

Phase 3: Watch for Completion Signal

The agent should eventually run pan done PAN-{ID}. Watch for:

# Check if completed marker exists
ls -la ~/.overdeck/agents/agent-pan-{ID}/completed 2>/dev/null

# What the forge says about the PR `pan done` opened
pan show PAN-{ID} --json | jq .

Expected state after completion:

• completed file exists in agent state dir
• A pull request exists, is open, and is not yet approved (reviewDecision
REVIEW_REQUIRED or null)
• GitHub issue has "In Review" label or status

Common failures at this stage:

• Agent doesn't call pan done — it just stops
• Agent calls it but dashboard doesn't process it (API error)
• Review isn't auto-triggered after completion

If review not triggered:

# Re-request review through the CLI (the trusted door)
pan review request PAN-{ID} -m "re-requesting review"

Phase 4: Monitor Review Agent

Once review is triggered, the review-agent specialist should wake up:

# Check specialist status
curl -s http://localhost:3011/api/specialists | jq '.[] | select(.name == "review-agent")'

# Watch review agent output
curl -s "http://localhost:3011/api/agents/agent-pan-{ID}-review/output?lines=50" | jq -r .output

# Check review progression on the PR itself
(cd "$WS_PATH" && gh pr view --json reviewDecision,reviews) | jq '{reviewDecision, latest: (.reviews | last)}'

Expected outcomes:

• reviewDecision: "APPROVED" → proceeds to test
• reviewDecision: "CHANGES_REQUESTED" → feedback sent to work agent

Common failures:

• Review agent not waking up (Cloister not running, specialist not initialized)
• Review passes but doesn't trigger test agent
• Review fails but feedback not delivered to work agent
• Review agent crashes mid-review

If review agent doesn't wake:

# Wake it manually
curl -s -X POST http://localhost:3011/api/specialists/review-agent/wake

# Or restart the review cycle through the CLI
pan review restart PAN-{ID}

Phase 5: Monitor Feedback Loop (if review failed)

If review returned feedback, the work agent should receive it and fix issues:

# Check if work agent received feedback
curl -s "http://localhost:3011/api/agents/agent-pan-{ID}/output?lines=50" | jq -r .output | tail -20

# How many review rounds this branch has been through (convergence signal)
ls "$WS_PATH/.pan/review" 2>/dev/null | wc -l

The work agent should:

1. Read the feedback
2. Fix the issues
3. Run pan review request PAN-{ID} -m "Fixed: ..."

If feedback not delivered: This is a code bug to fix. Check:

• send-feedback-to-agent skill
• Dashboard's feedback delivery mechanism
• Agent's message queue in ~/.overdeck/agents/agent-pan-{ID}/mail/

Phase 6: Monitor Test Agent

After review passes, test-agent should run:

# The test verdict: the artifact the test role writes, then the PR comment
cat "$WS_PATH/.pan/test/result.json" 2>/dev/null | jq .
(cd "$WS_PATH" && gh pr view --json statusCheckRollup) | jq '.statusCheckRollup'

# Watch test agent
curl -s "http://localhost:3011/api/agents/agent-pan-{ID}-test/output?lines=50" | jq -r .output

Expected: {"status":"passed"} in the artifact and green checks on the PR

Common failures:

• Test agent not triggered after review passes
• Tests fail but agent doesn't report results
• Test agent crashes

Phase 7: Verify Merge Readiness

After tests pass:

# Final check — the forge's own account of the PR
(cd "$WS_PATH" && gh pr view --json reviewDecision,mergeable,statusCheckRollup) | jq .

Expected final state: the PR is approved, its checks are green, and the
forge reports it mergeable. That conjunction IS merge readiness (PAN-3917 FR-9);
nothing stores it.

At this point, the user clicks MERGE in the dashboard.

Intervention Protocol

When you find a bug in the Overdeck infrastructure:

1. Log the finding — add to your running bug list with symptom, expected behavior, and actual behavior
2. Check for existing issues — gh issue list --state open --search "keyword" to avoid duplicates
3. File or update a GitHub issue — include context from this oversight run
4. Assess: blocker or not?
   • If blocker: Stop the agent, fix the code, rebuild (npx tsup or restart dashboard), resume the agent, continue testing
   • If not blocker: Log it, file the issue, keep going — don't derail the test run for non-blocking bugs
5. After fixing a blocker, verify the fix actually works by continuing through the same pipeline stage
6. Continue monitoring from the current phase

What Counts as a Bug

Be thorough. All of these are findings worth logging:

• Dashboard shows wrong status, stale data, or flickers
• Agent state file says one thing, dashboard shows another
• xBRIEF tasks not created, duplicated, or disappearing
• Specialist doesn't wake up or takes too long
• Terminal output garbled or missing
• API returns unexpected response
• Status transition skipped or out of order
• Workspace in inconsistent state (e.g., directory exists but no git worktree)
• Any error in server logs, even if the pipeline recovers

Key API Endpoints

Endpoint	Method	Purpose
/api/health	GET	Dashboard health
/api/agents	GET	List all agents
/api/agents/:id/output	GET	Agent terminal output (?lines=N; the Herdr or tmux pane, then output.log)
/api/agents/:id/activity	GET	Agent activity log
/api/issues/:id/approve	POST	Approve & merge
/api/specialists	GET	List specialists
/api/specialists/:name/wake	POST	Wake specialist
/api/cloister/status	GET	Cloister status
/api/cloister/start	POST	Start Cloister

Timing Expectations

Phase	Expected Duration
Work phase	5-30 min (depends on complexity)
Review	2-5 min
Feedback fix	5-15 min per round
Testing	1-3 min
Total (no issues)	10-40 min
Total (with 1 feedback round)	20-60 min