actively investigating a hypothesis — running a sweep, dispatching multi-agent analysis, designing serial adversarial gates,
Resources
1Install
npx skillscat add terrylica/cc-skills/crucible-investigation-methodology Install via the SkillsCat registry.
We need to produce a 2-3 sentence plain-text summary, objective, factual, no marketing, no superlatives, no calls to action. Must be at most 60 words. No quotes, no markdown, no bullet points, no headings. Just plain text. Summarize: skill is investigation methodology with 6 patterns: LLM-native data representation (quintile tokens), serial adversarial gates, etc. It helps agents investigate hypotheses by encoding data, running sweeps, multi-agent analysis, designing adversarial gates to validate edges. Problem: avoids false positives, ensures robust hypothesis testing.
Investigation Methodology — 6 execution patterns
Self-Evolving Skill: If any pattern here fails in practice (wrong results, wasted compute), update the section AND append to
references/evolution-log.md. Don't defer.
These 6 patterns executed in service of a-research-foundations. They are the "how" to the foundations' "why". Apply in roughly this order for a new hypothesis.
1. LLM-native data representation — quintile tokens
Before asking an agent to "look at" numerical market data, encode as per-bar token sequences using rolling quintile ranks within a causal window.
Canonical schema:
idx dir body_q range_q dur_q uwick_q lwick_q loc sess fwd+H...Each quintile is 1..5 in a causal 200-bar rolling window (see Skill A §1). Agents can spot motifs like +1:5:1|+1:5:1|+1:5:1 (three consecutive fast big-up bars) that are invisible in float-space.
Context-budget rule: 60 KB tokenized stats-table fits in agent context; 67 MB raw bars don't.
Full reference: findings/methodology/01-llm-native-data-representation.md.
2. Serial adversarial gates (A/B/C/D/E protocol)
Before trusting any in-sample positive, survive 4-5 independent gates in series.
| Gate | Question | Catches |
|---|---|---|
| A — Directional breakdown | Is edge from long/short/both? | Diffusive-looking edges that are actually directional |
| B — Mirror symmetry | Does the inverse trigger show mirror edge? | Sample-window drift inflating one side |
| C — OOS time-split (80/20 chronological) | Does finding survive on held-out later data? | In-sample overfit |
| D — Cross-asset replay | Does it replicate on other symbols? | Asset-specific overfit |
| E — Full-history per-year | Is it positive in ≥60% of years? | Single-year-luck |
Gate C is non-negotiable. Never report a positive without time-split OOS.
Classify verdicts:
- All pass →
validated-cross-asset(promote) - A+B+C pass, D fails →
validated-asset-specific(narrow scope, don't kill) - C fails → kill and archive with
resurrect_if:conditions
Full reference: findings/methodology/05-serial-adversarial-gates.md.
3. Multi-lens agent synthesis (4-5 parallel specialists)
When brute force is infeasible, launch 4-5 specialist agents in parallel with distinct analytical lenses. Convergence = validation; divergence = diagnostic.
Canonical lens set:
- Pattern-motif / n-gram
- Regime / trend classifier
- Morphology / wick hunter
- Information theorist / surprisal
- Hidden-signal hunter / skeptic critic — ALWAYS include; prevents confirmation bias
Rules:
- Same raw data, different lens prompts
- Each agent briefed: "test against shuffled-null, estimate multiple-testing burden, require null-test z > 2 AND per-year stability"
- 5th agent explicitly asks: "prior 4 agreed; do you concur or dissent?"
- ≥3/5 convergence = likely real; only 1 agent = likely lens-specific artifact
Disagreement is information. If two agents find different signals, test both independently.
Full reference: findings/methodology/04-multi-lens-agent-synthesis.md.
4. Per-trade enrichment for loss postmortem
When a signal works sometimes and fails sometimes, convert aggregate question → row-level question.
Pipeline:
Step 1 — Run signal across full history → collect N trade outcomes
Step 2 — Compute 20-30 causal features at each trigger bar
Step 3 — Emit parquet (per-trade × feature matrix)
Step 4 — Ship to multi-lens agents (§3)
Step 5 — Each agent hunts filters
Step 6 — Evaluate filters via shuffled-null + OOSMust-have feature categories (at least one from each):
- Trend/regime (SMAs, slopes, autocorrelations)
- Volatility regime (rv, range compression, Kaufman ER)
- Pattern context (quintiles at trigger + lags)
- Exhaustion/cluster (bars-since-last-signal, signal-count-in-window)
- Position (dist-to-swing-high ATR-normalized)
- Calendar (hour, weekday, month, session)
Session example: turned +0.178 bps baseline → +0.514 bps filtered (2.9× lift).
Full reference: findings/methodology/06-per-trade-enrichment-postmortem.md.
5. Agnostic-null cascade (orthogonal null retest)
When a signal passes one null type, retest with an orthogonal null before trust. Example from session:
- Phase F-B shuffled the vol-forecast column → +0.370 signal "lost" to z=−5.15 under that null
- Phase C shuffled the trigger mask (proper null) → +0.439 signal passed at z=+5.74
If Phase C had been the only test, we'd have trusted a potentially wrong null. If Phase F-B had been the only test, we'd have missed a real signal.
Rule: whenever a positive emerges, identify AT LEAST ONE orthogonal null and retest. Common orthogonal pairs:
- Feature-shuffle AND mask-shuffle
- Permutation AND block-bootstrap
- Shuffled-selection AND parametric (e.g., binomial tail)
If signal passes both, trust more. If it passes one but fails the other, investigate why — the divergence tells you something about the signal's mechanism.
6. Compute orchestration (pueue + wrapper scripts)
Long parallel sweeps go through pueue on BigBlack with a specific discipline.
Anti-pattern (silently fails, 0-second "successes"):
ssh bigblack '~/.local/bin/pueue add -- bash -c "VAR=1 python script.py"'Correct pattern:
cat > /tmp/run.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
export VAR="${VAR:-default}"
cd /tmp
exec uv run --python 3.14 --with numpy --with pandas python -u /tmp/script.py
EOF
scp /tmp/run.sh bigblack:/tmp/
ssh bigblack 'chmod +x /tmp/run.sh && ~/.local/bin/pueue add --group mygroup --label "job" -w /tmp -- /tmp/run.sh'
ssh bigblack '~/.local/bin/pueue wait <id> --quiet; tail -50 ~/.local/share/pueue/task_logs/<id>.log'Key points:
- Wrapper script (never inline
bash -c) — pueue'ssh -cwrapping strips inline quotes python -ufor unbuffered stdout (otherwise logs stay empty for 40 min)pueue restart --in-place <id>(not barerestart, which creates new task IDs)- Use
forkmultiprocessing context to share pre-computed arrays via COW
Full reference: findings/methodology/08-compute-orchestration-pueue.md.
See also: Skill(devops-tools:pueue-job-orchestration) for extended patterns.
Confirmation counts (provisional)
| Pattern | Confirmed | Notes |
|---|---|---|
| 1. quintile-tokens | 3 | Qualitative scan, Phase B stats, per-trade CSV |
| 2. serial-gates | 1 major (NGRAM3FU 4-gate pass) | Proven template |
| 3. multi-lens agents | 3 | Act-2 qualitative, Phase C stats, Phase L loss-postmortem |
| 4. per-trade-enrichment | 1 | Phase L — needs re-confirmation |
| 5. orthogonal null cascade | 1 | F-B vs C — the discovery of the principle itself |
| 6. pueue orchestration | 30+ tasks | Very high confidence; standard infra |
Post-Execution Reflection
After invoking this skill:
- Did a pattern catch a bug, save time, or produce a validated finding? Increment
confirmedin the table above; note the session/audit folder inreferences/evolution-log.md. - Did a pattern fail (misled you, wasted compute)? Demote in the table with a brief note; add to evolution log with link to the failing session.
- New execution pattern emerged? Draft a new section + append to evolution log.
- Compute anti-pattern caught you again? Re-check §6 wording — clarify the trap.