OpenClaw Testing

Proof policy

• PR CI = default broad proof. Change builds + focused tests pass → open
  ready PR. CI runs broad suites; never delay PR for pre-PR suite runs.

• Before PR, prove narrowly: touched tests with
  pnpm test <file> --maxWorkers=1, targeted typecheck, lint, format.
  Prefer local proof for trusted, bounded work when dependencies and current
  host capacity are qualified. Local execution is optional: a focused command
  can still load a large compiler graph or expand to broad lanes.

• Choose remote proof deliberately. Use PR CI for broad proof; Crabbox/Testbox
  may cover expensive checks, a busy or unqualified local host, another OS/device,
  live providers/channels, long E2E, or heavy benchmarks. Acquire only when the
  command is ready and capacity is granted promptly. If queued, stop only the
  task-owned allocation; use PR CI or a safe local scope. Keep uncovered
  platform-specific proof pending. Never force heavy work onto a busy machine
  or chain providers. Use the smallest profile proven for the selected workload.

• Never rerun or re-push just for green. Fix real failures; read available
  CI failure classification before attributing red.

• Prove changed contract, complete required gates, finish. Broaden/repeat only for
  changed inputs, failures, or named unresolved risk.

• Follow scoped AGENTS.md; authoring/reviewing tests: use
  boundary guide
  and test-audit.

Select the proof

• Select proof for touched contract; reuse valid proof (rerun for changed inputs or missing coverage); finish workflow's required gates within user/host limits; report unrun checks and gaps. Prove user-visible behavior through real flow when feasible; external APIs need live contract proof; isolated mock-Gateway harness = valid channel boundary proof. Docs-only: docs sanity and git diff --check.

• Runtime defect: reproduce real entry point; rerun it + relevant siblings.

• Trusted source: pnpm changed:lanes --json, focused tests + targeted typecheck/lint/format; pnpm check:changed = optional smart local check.

• Public SDK/plugin contract: representative consumers, not all-plugin sweep.

• Build output, lazy imports, package boundaries: include pnpm build.

• Workflow: git diff --check + pnpm check:workflows.

• Docs only: relevant docs/link/format checks + git diff --check; no runtime tests.

• Read on demand: local commands,
  Crabbox, OpenClaw remote setup,
  package/Docker proof,
  release CI,
  plugin release matrix,
  Docker authoring, channel skill/
  Control UI E2E.

• An isolated mock-Gateway harness is valid channel boundary proof; state live
  gaps. Prove user-visible behavior through the real flow when feasible;
  external APIs require live contract proof. Read the scoped channel/platform
  guide for live-channel or native-app work. Telegram claims require
  Test Server userbot proof with Convex-leased credentials.
  Platform claims require real-device evidence; macOS permission proof requires
  a stable signed app.

• Release proof: preserve candidate/Tooling SHAs; never grants publication authority.

Source and state boundaries

• Never execute untrusted contributor/fork code, wrappers, config locally.
  Use secretless fork CI or sanitized direct AWS through Crabbox; never
  credential-hydrated Testbox. Credentialed execution: maintainer approval after
  review required; never hydrate untrusted lease.
• Isolated state + free port. Never restart, edit, or test operator Gateway/real
  data without explicit per-task approval. Preserve unrelated processes and
  shared dependency installs.
• Concurrent test/check commands: separate OPENCLAW_VITEST_FS_MODULE_CACHE_PATH
  values or run serially; checks can run Vitest too. Repository wrappers only,
  never raw Vitest. Prepared linked worktrees: node scripts/run-vitest.mjs <file> --maxWorkers=1 avoids pnpm reconciliation; PR CI runs the broad checks.

CI failures

• PR CI fails → pnpm test:ci-failed <pr>; read pre-existing/new/unknown classification.
• Fix new failures; investigate unknown → pnpm test:ci-failed <pr> --run until selected tests pass locally → push once.
• --run enables OPENCLAW_NODE_TEST_PLAN_CONTINUE_ON_FAILURE=1: collect ordinary failures across selected configs, while infrastructure failures still stop; use the same opt-in for manual focused reruns.
• Pre-existing failures: failure-table landing rule. Matching signatures ≠ job-name-only attribution.
• Never gh run rerun to chase green; never local full suite for CI repair.
• Bind diagnosis to exact SHA + job. Fetch failed logs once; prefer exact run/job
  state over stale PR rollups.
• Distinguish product, harness, infrastructure, credentials, superseded-run
  cancellation. Snapshots pass macOS/fail CI: reproduce Linux/Node bytes before
  regenerating.

Baseline and expected-failure exceptions

Narrow baseline, snapshot, ignore, and expected-failure exceptions are implementer
decisions only when each entry is verified against source and the PR records its
reason (for example, tool-owned metadata in an upgrade snapshot while user data
stays exact). Never mask a real regression or user-data change. Blanket or
unverified exceptions need approval. Exact shrink-only ratchet updates are
maintenance.

Test failure policy

• Failures = defects. Bounded reproduction attempt (same shard order first);
  identify owner; fix product, fixture, shared-state, ordering bugs.
• Proven repair: add regression; cite another owner's fix when applicable.
• Safe fix cannot be established/completed: record original failure, attempts,
  evidence, uncertainty in PR; continue under normal CI/review gates.
• Unresolved failure alone neither blocks landing nor requires extra approval.
  Passing replay ≠ proven fix.
• Never hide failures with retries, longer timeouts, weaker assertions, broader
  mocks, altered baselines.