Run a coordinated multi-session research program only when explicitly requested; maintain a logbook and durable snapshots.
Install
npx skillscat add marin-community/marin/run-research Install via the SkillsCat registry.
Multi-session research
Long-lived work should leave a durable record: a logbook and enough
commands/config to reproduce results. When the work has a coordinating issue,
keep its summary current. Do not publish secrets or private work the user asks
to hold back.
Use background-research, task-logbook, wandb-reporting, task-snapshot,
and update-docs for their named artifacts. Add a domain skill only when the
research directly requires its workflow.
Core Artifacts
- A logbook at
.agents/logbooks/<topic>.md. - A living hypothesis queue in the logbook, derived from append-only entries
and updated as hypotheses are proposed, blocked, falsified, or promoted. - A long-lived branch, for example
research/<topic>orresearch/<user>/<issue>-<topic>, with the logbook, research code, configs,
small artifacts, and test harnesses needed to reproduce results. - One or more commit or tag snapshots for meaningful milestones.
- Often a "production" branch that gets PR'd and merged.
Standard Workflow
1. Prologue
- Keep an existing user-specified branch; otherwise create or use a long-lived
research/<topic>orresearch/<user>/<issue>-<topic>branch. - If a coordinating issue exists, link it bidirectionally with the logbook.
- Choose one short experiment ID prefix and use IDs such as
MOE-HC-001in
entries, runs, and comments; use two to four shared tags. - Record the goal, success and stop criteria, baseline, initial hypothesis
queue and experiment matrix, relevant code, and references.
2. Research Loop
- Forage: gather prior work and local context.
- Propose: update the living hypothesis queue and pick the next test.
- Run: implement the smallest useful experiment and collect evidence.
- Interpret: compare against baseline, decide confidence, and update the
logbook. - Promote: move decision-relevant claims into the current summary and a
coordinating issue when one exists. - Seal: snapshot durable results or extract production work.
2.1 Forage: Background Research
Use background-research at the beginning, after a significant change of
direction, or whenever you hit a wall. Assume medium or low effort unless
the decision is expensive.
Use the background-research output to update the logbook's hypothesis queue:
add new candidates, revise weak ones, mark known dead ends, and promote
well-supported ideas into the next experiment matrix. If a coordinating issue
exists, let background-research and task-logbook decide what belongs there.
2.2 Propose / Run / Interpret: Dev Work and Experiments
For research-branch dev work, optimize for learning speed while preserving
operational security and cost controls. Ad-hoc scripts, temporary config knobs,
and copy/paste are acceptable there. Production-facing code keeps the usualAGENTS.md quality bar.
For each non-trivial experiment:
- Do the dev work needed for the experiment.
- Run the benchmark or experiment. Use the
use-irisskill for long-lived runs. - Append exact commands, config, key outputs, interpretation, and next decision
to the logbook. Followtask-logbookfor issue updates when an issue exists. - Push dense scalar series, plots, or raw artifacts to W&B or another store
when they are too large for issue comments.
3. Epilogue: Seal
Seal when requested or when the defined goal is reached.
- Update the logbook's final TL;DR, conclusion, decision log, and
negative-results index. - If a coordinating issue exists, update its body and add a final comment
covering what worked, what did not, confidence, limitations, and ordered
next steps. Followtask-logbook. - Use
update-docswhen behavior, operational practice, reusable guidance, or
durable research findings changed. - Ensure the final logbook entry and snapshot links are present.
- Close the coordinating issue when one exists and the research thread is
complete.
If the research produced useful production changes, extract them into a clean
branch that links the research record without including it. Before finishing,
update the TL;DR and conclusion, list next steps, and link the final snapshot
and production PR when one exists. Add the same links to the coordinating issue
when one exists.
Practical Rules
- Prefer short-lived code changes unless a persistent harness is clearly useful.
- Keep benchmark harnesses configurable and minimal.
- Record exact command lines for every headline number.
- Treat failures and negative results as first-class data. Record dead ends and
excessive hyperparameter sensitivity; skip routine bugs or undertuning.