kirodotdev

writing-tests

"Kiro Crew repo only: how to write a backend pytest test with NO side effects that does not flake. Use when adding, editing, reviewing or debugging a test there: which conftest applies, what leaks (temp dirs, data home, ~/.kiro, cron, threads), the seven flake classes, cross-platform traps."

kirodotdev 4,973 1,058 Updated 2w ago
GitHub

Install

npx skillscat add kirodotdev/kirocrew/writing-tests

Install via the SkillsCat registry.

SKILL.md

Writing a Kiro Crew test that does not leak and does not flake

Scope guard: this skill applies ONLY to the Kiro Crew source repository (or a
worktree of it). Its rules are conventions of this repo's suite. In any other
project, ignore it.

The canonical, longer reference is
`docs/system-specs/common/testing-conventions.md`.
If this skill and that document disagree, the document wins. What this skill adds is
the decision order — what to check first, and what the failure looks like when you
get it wrong.

Where this skill stops, and which sibling takes over

This skill owns authoring a test: isolation, determinism, cross-platform
behaviour, diagnosing a residue failure, and suite speed. Three siblings own the
neighbouring steps, and none of them restates what is here:

You are doing Skill
Writing, fixing, or speeding up a test this skill
Running the build gate in a worktree, and deciding whether a red is yours kirocrew-worktree-dev
Driving a branch to a review-ready PR and through CI kirocrew-prepare-pr
Polling that PR until it is green babysit

The line that matters most in practice: kirocrew-worktree-dev tells you whether a
failure is yours (re-run it on origin/main, mine CI for what is genuinely flaky);
once it is yours, the fix is here. Do not fix a flake from a summary of the
determinism classes — pick the class from the symptom, in Rule 2.

The two properties, and why they are one problem

A test must be hermetic (no effect that outlives it, anywhere but its own tmp dir)
and deterministic (same verdict every run, on every platform, in any order). They
are the same problem because the suite runs -n auto --dist loadgroup: a side effect
is not just untidy, it is the input to another test on the same worker, and it
surfaces as a flake in a file you never touched.

Rule 0 — Know which conftest is under your file BEFORE you isolate anything

setup.cfg declares two testpaths and they do not get the same fixtures. The
table is in testing-conventions.md
§ Which conftest you are standing on — read it rather than a second copy here, which
would drift. The short version: only test/ gets test/conftest.py;
src/kiro_crew/apps/builtins/**/tests/ (and the one **/*_tests/ suite) gets the
rootdir conftest.py plus that suite's own conftest.py where one exists.

The rootdir conftest.py is the host floor — the guards that protect the
developer's machine, so they hold everywhere. That covers damage (temp dirs, the data
home, services) and resource exhaustion: the xdist worker budget is registered there,
from the repo-root xdist_budget.py, because a run that spawns one worker per core on a
small laptop takes the machine down and does not care which testpath asked. Everything
else is in test/conftest.py.

Getting this wrong is the most expensive mistake in this suite, because it fails
silently and asymmetrically: an in-package test that assumes test/conftest.py's
fixtures passes on CI (where the operator's home is empty) and damages a real install
locally. When you add isolation, ask: could a test in ANY testpath damage the host —
or consume enough memory, cores or disk to take it down — without this?
If yes it
belongs at the rootdir; if no, put it in test/conftest.py, where the in-package suites
pay nothing for it.

Rule 1 — The side-effect floor: what actually leaks

Work down this list. Each item is a real leak that has happened here.

1a. Temp directories — register the destruction in the SAME scope

Prefer tmp_path. If you need mkdtemp(), register cleanup with addCleanup on the
next line — never an rmtree in tearDown, because unittest skips tearDown
entirely when setUp raises, so that shape leaks on exactly the failing runs nobody
watches. The before/after example is in testing-conventions.md § Rules.

ignore_errors=True also hides a cleanup that could not finish, so it is not proof
of anything. The floor helps, but it is not your discipline: the rootdir conftest redirects
tempfile's base per run, removes it, and reports residue. CI's Backend Tests* jobs
set KIROCREW_TMP_RESIDUE_STRICT=1, so a leak of yours turns that shard red; a local run
only warns unless you set it. Need a data home? Use the conftest's per-test KIROCREW_HOME
pin, not a private temp home: a background worker can re-create a home you removed.
(Details: testing-conventions.md § Rules.)

Running the suite from a Kiro Crew agent needs no setup. The agent's shell inherits the
live KIROCREW_HOME, and the rootdir conftest sees KIROCREW_SCRATCH beside it and gives
the run its own home under $TMPDIR (the agent's scratch, reclaimed by the gateway)
instead of refusing. Do not make a home under /tmp/<name>, and do not fake HOME: a
fake HOME makes uv, npm and pip rebuild their caches under it, 10k+ inodes per run, and
nothing an agent may run can delete it afterwards. If a harness must fake HOME, keep
the caches shared: XDG_CACHE_HOME=<real home>/.cache UV_CACHE_DIR=<real home>/.cache/uv npm_config_cache=<real home>/.npm on the commands the harness itself runs. The conftest
keeps only its own bytecode and hypothesis caches on the real home.

Why it earns a guard: /tmp is often a tmpfs with a fixed inode budget
(1,048,576 on the hosts this was measured on) and it returns ENOSPC to every other
process on the machine while 90% of the bytes are free. It is not a tidiness issue,
it is a "your shell stops working" issue.

1b. The operator's data home, and ~/.kiro, which is a DIFFERENT axis

KIROCREW_HOME is pinned per test at the rootdir, which is what makes config_dir()
safe — and it needs to be, because resolving it is not a read: it creates the home
and its marker, and can run the ~/.kirocrew → ~/.kiro/crew migration.

Six shapes escape the env var:

  • monkeypatch.undo() in the body of a test. It reverts EVERY record on the shared
    instance — your own patches, every fixture that set something through it (stores
    pins KIROCREW_HOME that way), and until the floor got its own instance, the floor
    itself. A restore that ran empty_trash() after undo() emptied the operator's real
    trash. A patch you need to drop before the test ends goes in
    with pytest.MonkeyPatch.context() as patched:; undo() is not a scoped tool.

  • A detached task. asyncio.create_task(...) at boot with no one awaiting it keeps
    running after the test that started it has returned; when its work resolves a path
    (peek_next → spool_path() → data_home(), on a to_thread worker) it resolves
    the OPERATOR's home. Fixed where the task is scheduled: _start_channel_transports
    resolves spool_path() on the loop and hands it to the task, so the worker reads a
    location fixed at boot. Resolve every environment-derived input of a detached task on
    the calling side before detaching, or give the test a handle it can await.

  • What a closed loop leaves behind. pytest-asyncio 0.21 ends the loop with a bare
    loop.close(). A pending task's coroutine then gets GeneratorExit at garbage
    collection, so its finally blocks run then (_run_chat's queue cycle reaches a
    sync KiroCrewConfig.load()); a default-executor job (to_thread) is abandoned
    mid-flight. Both resolved config_dir() after the unpin and grew a fresh fake
    HOME's ~/.kiro/crew. The floor's tryfirst teardown hook now cancels pending
    tasks, runs them to completion and joins the executor while the pins hold, the way
    asyncio.run shuts down. A coroutine ... was never awaited warning pointing at that
    hook means your test left an unstarted turn behind: await it, or keep the patch that
    was meant to catch it in force until it has run.

  • A default that bypasses the pin on purpose. PodConfig.load() derives the pod
    plane from _default_home() / Path.home() so a pod cannot redirect the host's
    registry; the floor pins KIROCREW_POD_ROOT / KIROCREW_POD_ENV_DIR for that reason.
    A new resolver with its own default gets a floor pin AND a ratchet in
    test_host_isolation_floor.py in the same change.

  • Import-time from config_dir() (subagent_persistence._SUBAGENTS_DIR): the var
    is read after the module captured the path. Each has its own autouse pin. A
    collection-time probe is the same shape one step earlier: a module-level
    probe used by a skipif (_sandbox_can_spawn() in test/test_app_backend.py)
    runs before any pin and would read the REAL config.json; run such probes under
    an empty KIROCREW_HOME.

  • Import-time from Path.home(): ~/.kiro is kiro-cli's home, machine-wide and
    shared with the real installed agent — ~/.kiro/settings/mcp.json is the live
    agent's MCP server list. _isolate_shared_kiro_paths redirects these from a table,
    and test/test_host_isolation_floor.py fails when src/ grows a new one. That
    ratchet covers IMPORT-TIME bindings only. The lazy resolver
    config.paths.kiro_home() — and so kiro_agents_dir() / kiro_sessions_dir() — is
    yours to isolate, with KIRO_HOME or by patching Path.home(); they are not
    interchangeable, because the env var outranks the other. A test that skipped this
    projected the developer's installed agent specs and failed with an
    AcpRuntimeError naming a prompt file in an unrelated worktree.

    Some entries are excluded, and for two opposite reasons — already redirected
    elsewhere (the macOS launchd set, moved as one group by _isolate_launchd_paths),
    or a security anchor that must never move: the file browser's allow-list root
    file_explorer/server._HOME. Stub the reader, never
    move the anchor.
    Redirecting a matcher so a test can pass makes it assert against a
    pattern that no longer matches the thing it protects. An anchor that stops being an
    import-time Path.home() binding leaves this tripwire's reach entirely —
    kiro_usage_api's kiro-cli sqlite tuples now resolve the home inside
    identity_stores.sqlite_dbs(), so their anchor rule is pinned by
    test_identity_stores.py::TestUsageTuplesAnchorTheRealHome instead. Read
    _EXCLUDED for the live list rather than a copy here.

1c. A child process inherits pytest's CWD, which is the repo root

This is the leak a reviewer cannot see: no line says open(..., "w"), the write
happens in a grandchild, and the test can assert against tmp_path and pass while the
artifact sits in the checkout. An empty file produced this way has been committed and
shipped from this repo.

  • Pass cwd=<a directory under tmp_path> to any child that MAY create a file, and to
    every helper that spawns one.
  • Scope the assertion to where that child's CWD actually is. An assertion over
    tmp_path proves nothing about a child that ran somewhere else — and a security test
    whose payload escapes its own assertion is worse than no test, because it reports a
    guarantee it never checked.
  • A read-only query is exempt, and sometimes must be: git check-ignore against the
    checkout is asking a question about the checkout.

The rootdir conftest fails the run on new non-ignored entries at the repository root,
which is how this announces itself.

Two more things a spawning test owes, both learned from processes that outlived the run
by six days, spinning at 1464% CPU between them:

  • HOME and PATH are a PAIR. If you hand a child a fabricated environment,
    substituting one while inheriting the other is the defect. An inherited PATH on a
    developer host routinely leads with a version-manager shim directory (mise, asdf,
    pyenv, volta, nodenv), and a shim resolves its tool set from HOME — so with a
    substituted HOME the bare name python3 or node reaches the MANAGER, which finds
    no tool state and never execs anything. It spins, forever. Pin the real interpreter's
    own directory first on PATH, or resolve the tool to its real executable before
    building the env. Do not drop the HOME substitution instead: it is usually a
    blast-radius bound somebody chose on purpose.
  • Reap the process GROUP, on every exit path. A child is routinely a wrapper that
    forks, so Popen.kill() reaps the wrapper and leaves the real work running — and a
    bounded wait() ending in a bare pass then reports success. Use
    start_new_session=True + os.killpg on POSIX and CREATE_NEW_PROCESS_GROUP +
    taskkill /T /F on Windows; test/installer_test_helpers.run_bounded is the
    reference. A try/finally around the whole post-spawn body, not just the timeout
    branch, is what makes "every exit path" true.

1d. Background lifecycle — the one that beats every filesystem cleanup

A singleton with a daemon thread cannot be cleaned up by tidying files. The worked
example is sel.py: SecurityEventLog is a process singleton, its writer is a daemon
thread, and _init_locked binds the directory once from whatever _default_dir()
resolved then. So the first test to call sel() fixes it for the whole worker, the
thread keeps writing after that test ends, and _flush_batch opens with
mkdir(parents=True, exist_ok=True) — re-creating the directory after tearDown
deleted it
. MEASURED: exactly one stray directory per run of the ops-mission-control
suite. Full telling, with the stack it came from:
testing-conventions.md
§ Rules.

The fix was not better cleanup. It was giving the singleton a session-scoped
directory belonging to no individual test. When you touch a subsystem with a background
worker, ask: which directory did its thread capture, and does anything delete that
directory underneath it?

The same shape, one level up: a stub that replaces a shutdown/close/stop is
not a stop.
Three tests needed to observe that the metrics provider's shutdown
was called, so they replaced it with a recorder — and the real shutdown is what stops
OpenTelemetry's exporter thread. That thread then survived for the life of the worker,
and because the OTel SDK reinstalls it in every fork child via os.register_at_fork,
the sandbox's userns probe forked a MULTITHREADED child. unshare(CLONE_NEWUSER)
implies CLONE_THREAD, which the kernel refuses with EINVAL unless the caller is
single-threaded, and EINVAL is indistinguishable from "no CONFIG_USER_NS" — so the
worker cached "this host has no sandbox backend" and every later sandboxed spawn failed
closed. 19 red tests in two app suites, each passing alone, none of them a metrics test.
Spy and delegate; never replace a lifecycle method. The rootdir conftest now fails
the test that leaves an exporter thread running.

Two general lessons worth carrying out of that one: a thread whose target is a bound
method keeps its own object alive, so dropping references never collects it; and
anything registered with os.register_at_fork runs inside os.fork(), before it
returns, so a fork child is not reliably single-threaded.

Related traps in the same family:

  • A MagicMock config reads TRUTHY. Patching KiroCrewConfig.load with a bare
    MagicMock makes cfg.telemetry.enabled truthy, which starts a real recorder and a
    reader thread, and resolves Path(cfg.local_dir) to a relative path that writes
    into the repo. That is why KIROCREW_TELEMETRY=0 is forced for the whole suite. Any
    subsystem gated on a truthy attribute read off a config object has this shape.
  • A sleeper child that outlives the test. The suite spawns
    python -c "import time; time.sleep(30)" to simulate a hung process. Put the
    kill/wait in a finally or an addCleanup, never only on the happy path.
  • A fixed port. Bind port 0. A fixed number collides across xdist workers and
    with the operator's running gateway.

1e. Never leave the process working directory somewhere else

The CWD is per-PROCESS, so under xdist one test's os.chdir becomes every later test's
starting directory on that worker. Use monkeypatch.chdir, which reverts itself.

This is the leak with the widest blast radius measured here. Because a passing test's
tmp_path is removed at its own teardown, a test that chdirs into tmp_path and does
not come back leaves the worker in a deleted directory, and Path.cwd() then raises
FileNotFoundError for every later test that reaches it — including from inside
production code. The measured instance is in
testing-conventions.md
§ Rules; the shape to recognise is that it reads as "the suite is flaky" — many files,
each passing in isolation. The rootdir conftest restores the CWD before any fixture
teardown, but write the test so it would not need to.

1f. Never register a real cron job, and never touch a real service

The rootdir conftest traps the stdlib spawn funnels and refuses a
systemctl/launchctl invocation carrying a mutating verb. Read-only queries
(show, cat, is-active) are allowed and need no stub. A test reaching the make-live
cutover path must stub both _run_cmd and _dropin_path.

Rule 2 — Determinism: seven classes, one correct fix each

Never "fix" a flake with a rerun, a longer sleep, a weakened assertion, or a skip.
Full detail and examples: testing-conventions § Determinism. Its short form, twelve
MUST rules, is the
Determinism contract
at the top of that document, and the checklist below opens with the same twelve.

The seven classes, the tell that identifies each, and the ONE correct fix for each are in
testing-conventions.md
§ Determinism. Read the section matching your symptom before changing anything: most of them
have a fix that looks like the obvious one and is not.

What this skill adds is when to go looking — a test that passes alone and fails in the suite, or
one that splits by Python version rather than by machine load, is one of those seven and not a
mystery. Do not reach for a rerun, a longer sleep, a weakened assertion, or a skip.

The seventh class is the one a sleep hides best: two stamps written back to back can be
EQUAL.
On Windows through Python 3.12 time.time() steps ~15.6 ms, like
time.monotonic(), and some filesystems store mtimes in whole seconds, so a sleep
between two writes makes a tie less likely, never impossible. Set the stamps (os.utime(ns=...), a stepped
clock, an explicit created_at), assert strictly, and pin a time-sorted output's
tie-break with an equal-stamp pair.

Prove a test rather than trusting a green run: 20 repeats at -n0, 10 at -n 4 beside
its neighbours, and a shuffled order. A FIX to a flaky test also shows the forced
condition (a coarse clock, a late executor, a delay at the named seam) red on the parent
commit and green on the fix (testing-conventions § Proving a determinism fix).

The sixth class has a tell of its own: the run ends early, not red. On Windows a test that
blocks past --timeout is not failed, its xdist worker is killed, and with
--max-worker-restart=0 the run aborts with every uncollected result missing. If a full run
reports a few thousand tests instead of 100k+, look for worker ... crashed while running in the
log: the named test is one that can wait forever. Wait on the observable state, not a guessed
sleep, and bound the await whose refusal is under test with asyncio.wait_for so a missed
refusal fails by name at that line.

An adjacent trap: a patch target that misses. Patch the namespace whose
globals the code under test actually reads. It fails in both directions — patching a
package re-export when the caller reads its own defining module, or patching
pkg.mod.fn when the caller did from pkg.mod import fn and holds its own binding.
Either way the real function runs, the assertion passes for the wrong reason, and the
test pays real time. Treat an unexpectedly slow "mocked" test as evidence the mock
missed.
The opposite trap is a patch that is too WIDE: a side effect on a module global
also fires for every background worker that calls it (the eventlog-io legacy fold calls
members.read_dm_binding). Patch the function awaited in the window you mean, and pin
the order of the calls the test relies on.

Another: the host is an input, and a "surely-unused" number is not a constant.
999999 reads as an impossible PID and is not — pid_max is 4194304, so on a
long-running host it names a live process. That broke two tests in opposite ways: one
stopped pruning an entry whose owner "must be dead", and one accused a planted ps shim
of executing when it had not. If the code probes the PID, pin the probe; if the number
must never appear in real output, pick one no OS can allocate (99999999999).

The host's free MEMORY is an input too, and the most-used probe of it is
SubagentManager.spawn, which queues — registering nothing in _tasks — on a
pressured machine. A queued spawn is still a SubagentInfo, so the test dies a line
later on a bare KeyError, not on the assert that would have named the cause. Any
file driving spawn takes pytestmark = pytest.mark.usefixtures("healthy_host_memory"),
and test_subagent_spawn_host_pin.py fails when a new one does not. The guard it
pins, and why it stays transparent for the parser's own tests, are in
testing-conventions § Determinism 1. Both the fixture and its ratchet are
test/-only — healthy_host_memory lives in test/conftest_opt_in.py, bound by
test/conftest.py, and test_subagent_spawn_host_pin.py scans test/*.py
non-recursively. An in-package app suite cannot request the fixture and is not
swept, so a test there that drives SubagentManager.spawn must pin the floor reading itself.

One more, for tests of a single-flight or coalescing path ("N concurrent readers
share ONE scan"): the property only holds for readers that arrive WHILE the shared
operation is in flight, so the test has to keep it in flight until they have. An
instant stub does not: asyncio.gather starts the leader first, its executor job
finishes on the pool thread before the loop reaches the await, and on Python 3.13
the wrapped future is already done — awaiting a done future does not yield. The
leader then completes with a waiter count of one and the next reader assembles again,
once in five loaded runs. Gate the stub on a threading.Event, poll until every
reader is registered, then release it. Whatever the count of assemblies asserts, the
test must first establish the concurrency it is asserting about.

Rule 3 — Cross-platform: macOS, Linux (x86_64 + arm64), Windows

  • Route POSIX calls through platform_compat. See docs/system-specs/common/platform-compat.md. Most
    important: os.kill(pid, 0) TERMINATES the target on Windows — it is not a
    liveness probe. Use platform_compat.pid_exists.
  • Path length is a real constraint. Windows caps a path at 260 characters unless
    long paths are enabled, and a macOS AF_UNIX sun_path is capped at ~104 bytes —
    which a macOS basetemp alone already exceeds. That is why
    test/tmpdir_helpers.short_tmp_base() exists, and why anything that prefixes every
    temp path in the suite has to be measured, not assumed.
  • Case-insensitive filesystems. macOS and Windows are case-insensitive by default,
    so a test asserting that two paths differing only in case are distinct is broken
    there.
  • Windows clock granularity. Through Python 3.12 time.time() and
    time.monotonic() step ~15.6 ms, and so do Event.wait, lock and queue timeouts;
    only time.sleep has been high-resolution since 3.11 — so a short sleep can put two
    writes on one stamp, and "two writes have different timestamps" is a flake there
    (class 7).
  • Probe, do not guess the platform. test/conftest.py::_can_create_symlink is the
    model: creating a symlink needs SeCreateSymbolicLinkPrivilege, which CI runners
    hold and an ordinary shell does not, so a blanket skipif(IS_WINDOWS) would drop the
    assertion exactly where it needs to run. Use the make_escaping_link /
    make_dir_link helpers, which fall back to a junction.
  • arm64 vs x86_64 rarely matters for test logic, but check it when you touch memory
    or page arithmetic (SC_PAGE_SIZE is 16K on some arm64 configurations) or a
    dependency with per-architecture wheels.
  • Windows gaps are tracked as burn-down lists, not scattered skips:
    test/windows-collect-ignore.txt and test/windows-expected-failures.txt. Anything
    NOT listed still fails the job. Delete a line when you fix its test.
  • Never --deselect a whole file for a missing host capability. Guard the test that
    needs it — skipif(not userns_available()) for the OS sandbox — so the report names the
    capability. A deselect is invisible in the output, takes the file's other tests with it,
    and never goes red when its reason expires: eleven such files kept 608 tests off every
    PR, of which 523 would have run on all three platforms, long after the cost that
    justified the exclusion had been fixed.

Rule 4 — Diagnosing a residue failure

The residue report runs in a session-fixture teardown, so it is attributed to the last
test the worker ran
, which is almost never the culprit. The guard carries its own
bisector — use it instead of guessing:

KIROCREW_TMP_PER_TEST=1 pytest src/kiro_crew/apps/builtins/<app>/tests -n0 -q
# AssertionError: 1 temporary entry outlived this run under /tmp/kc-pytest-you-951504:
#     test_provider_listing_never_contains_a_token/tmpw2kvty2z

Each residue name becomes <test id>/<leaked name>. If the leak survives a tearDown
that visibly removes it, suspect Rule 1d: something re-created the path after
cleanup. Confirm by wrapping os.mkdir in a throwaway -p plugin and printing a stack
for paths under the run's temp root — that is how the sel-writer thread was found.

For a repository-root residue failure, the cause is almost always Rule 1c: a subprocess
spawned without cwd=.

Rule 5 — Keep the parallel suite fast

At 100k+ tests, per-test setup cost dominates any single slow test — an autouse
fixture is paid once per test, over 100,000 times. Profile, never guess; compare candidates back to back on
the same host (run the change, then the base in git worktree add --detach <dir> <base>;
never git stash: every worktree shares one stash list), because a loaded host makes an
absolute number meaningless.

The recurring wins, in order of leverage:

  1. An autouse fixture that costs more than it protects — one requesting a
    tmp_path it never uses; a repeated tmp_path_factory.mktemp (it scans the whole
    basetemp to pick its suffix, so it slows as siblings accumulate); an unconditional
    mkdir where the consumer only needs a path. Measure the whole chain against a
    file of trivial assert True tests.
  2. A production timeout or poll the test never asserts on — monkeypatch the
    interval to 0; the branch still executes, only the waiting goes.
  3. An expensive immutable thing built per test — a real git repo costs ~1–1.6s in
    subprocesses. Build it once scope="session" and copytree it per test; copy from
    the template rather than yielding it, so nothing one test does can reach another's.
  4. A module-cached tree scan paid once per WORKER — an lru_cached rglob +
    ast.parse of src/ (~30 s) is computed again on every xdist worker the module's
    tests land on; five full runs measured eighteen ratchet modules each re-scanning on
    3–5 workers, ~22 CPU-minutes per run. Add
    pytestmark = pytest.mark.xdist_group(name="tree_scan_<module>") — one group per
    file, never one shared group — and the scan runs once per run. The group also
    carries the tree-scan timeout: test/conftest.py gives every test in a tree_scan
    group with no timeout of its own 300 s instead of the suite's 120 s.

After any speedup, mutate the production code the test covers and confirm the test
still FAILS.
A test made faster by checking less is a regression. Restore from a copy
of the file you mutated, not from git — git checkout -- discards unrelated uncommitted
work — and sequence with ;, not &&, or the restore only runs when the mutation
did not work.

Rule 6 — MEMORY is the other budget, and collection is most of it

A worker costs ~2.0 GiB (measured median at -n 12; worst observed 2.8 GiB), and
~1,499 MiB of that is paid before your test runs: every xdist worker independently
collects every item in both testpaths (106,491), and 99% of that footprint is private,
so more workers never amortize it. This is why -n auto is bounded by available memory
— on an 8–16 GiB laptop the full suite otherwise swaps the machine.

Both halves of that model doubled between audits (from ~57k items / ~750 MiB), so
re-measure rather than trusting the numbers above once the suite grows by half
again: --collect-only -n0 reproduces a worker's collection peak. The reservation in
xdist_budget.py is sized on them, and an under-sized reservation is how a laptop
starts swapping.

The consequence for how you write a test:

  • An allocation at module scope is multiplied by every worker, and outlives the
    test.
    A big literal inside @pytest.mark.parametrize is the worst shape: it is
    built while the module is IMPORTED and the mark keeps it alive on the function
    object for the whole session, so one test's payload is charged to all of them. Two
    such literals — a 64 MiB frame and a 40 MB string — cost 102 MiB per worker until
    they were moved into the test bodies behind a sentinel.
  • A transient peak counts too, because a worker's high-water mark is what the
    budget must reserve. Building an image as a list of per-pixel tuples cost ~390 MiB
    for a 17 MB PNG; one frombytes over a bytes buffer was 10x smaller and
    byte-equivalent.
  • Derive a size from the production constant rather than restating it. A literal
    one past a production size ceiling, restated beside that constant, hides both the
    coupling and the cost, and goes stale silently.
  • Suspect a module-scope literal whenever a file's collection RSS is large; measure
    it with pytest <file> --collect-only and /proc/self/status's VmHWM.

Checklist before you push a test

Determinism top 12

One box per rule of the Determinism contract,
naming the shape; the contract line holds the detail and its numbers. The helpers are in
kiro_crew.testing (clock, wait, ids). Every other item below is a specialised trap.

  • D1: wait on the asserted state with a raising, bounded poll (wait_until,
    async_wait_until, until_parked), never a sleep
    (class 2)
  • D2: one clock, installed on the module-under-test's own binding
    (ManualClock.install, the manual_clock fixture)
    (class 2)
  • D3: set the timestamps (ManualClock(tick=...), os.utime(ns=...), seq_ids),
    assert order strictly, pin the tie-break
    (class 7)
  • D4: no asserted order the code does not define
    (class 7)
  • D5: the product's zone and a frozen instant, never the host's (local_tz)
    (class 1)
  • D6: seeded RNGs and unallocatable fake PIDs (seeded_rng, unallocatable_pids)
    (class 1)
  • D7: no upper bound on a measured duration beyond what the contract allows
    (class 5)
  • D8: every self-unblocked await, join or communicate bounded, failing by name
    (class 6)
  • D9: listeners bind port 0 and report the port (Rules)
  • D10: loopback only, the network stubbed at the product's seam
    (side effects)
  • D11: globals through monkeypatch, no in-process reload, evictions restored
    (class 4)
  • D12: proven by repeats, a shuffled order and, for a fix, the forced condition
    (proving a fix)

Specialised traps (search when your test touches X)

  • Nothing outlives the run: no temp residue, no write to ~/.kiro or the real data
    home, no cron job, no service change, no file in the checkout
  • Every mkdtemp has addCleanup on the next line (or uses tmp_path)
  • Every child that may create a file gets cwd= under tmp_path, and every
    assertion is scoped to where that child actually ran
  • Anything whose default directory is Path.cwd() (TaskRunner(work_dir=...)) is
    constructed with an explicit tmp_path; a gitignored name at the repo root is the
    one leak the residue guard cannot see
  • A background worker gets every environment-derived input (paths AND config) from the
    dispatching thread, and registers itself so the rootdir conftest teardown can join it
  • Every thread, task, child process, socket and connection it starts is stopped in a
    finally or an addCleanup
  • Globals mutated through monkeypatch, never raw assignment — including os.environ
    keys a REAL production startup path is known to write (PLAYWRIGHT_MCP_OUTPUT_DIR,
    KIROCREW_TELEMETRY, PATH), restored in the shared helper that drives it
  • A variable the code under test WRITES and that is absent before the test goes
    through forget_env_at_teardown(monkeypatch, name) — delenv(raising=False) on an
    absent key records no undo, and a delenv AFTER the write puts the written value back
  • No monkeypatch.undo() in a test body: it reverts every fixture's records too. A
    patch you need to drop early lives in with pytest.MonkeyPatch.context() as patched:
  • No module-level skipif probe that reads KiroCrewConfig / config_dir() /
    Path.home() — it runs before any pin and observes the operator's real config
  • A collection-time probe that can construct a process singleton (sel(), a config
    loader) RETIRES it before its temp home is removed — the session floor only resets
    singletons at the first test's setup, and a live one re-creates the deleted path
  • Every MagicMock attribute the code under test converts or compares (int(),
    len(), bool(), <) is set explicitly in the mock helper — int(MagicMock())
    is 1, and a drain loop reads that as "one still pending" until its deadline. A
    test whose duration equals a production timeout has hit this
  • A complexity guard (ReDoS, "stays linear") is sized so the regression it exists to
    catch FAILS it inside --timeout rather than hangs the worker: RAMP the pump one
    unit at a time on thread CPU (assert_rejected_without_backtracking), never a single
    huge input on a wall clock — no fixed "small" size is safe against every growth rate
  • A test of a refused location names one the guard refuses on THIS host: /usr is
    C:\usr on Windows — accepted by the resolver, and then created on the system drive
  • Fixture paths are absolute on EVERY host: host_abs("usr", "bin"), never a /usr/bin
    literal (ntpath.isabs rejects a driveless path from Python 3.13); a path that belongs
    to a simulated platform is judged with that platform's module (posixpath)
  • Nothing assumes the ancestry of tmp_path is bare (no .venv, no project marker
    above it), that 127.0.0.1:1 refuses connections, that python3 is on PATH (spawn
    sys.executable), or that git/gh sit in a trusted system directory
  • A fabricated child environment does not substitute HOME while inheriting PATH
    (or vice versa): with a shim-led PATH the bare python3/node is a
    version manager that finds no tool state under the new HOME and spins forever
  • Every child that could outlive the test is reaped by process GROUP
    (start_new_session=True + os.killpg; taskkill /T /F on Windows) from a
    try/finally around the whole post-spawn body, not only the timeout branch
  • A skipif on an external tool gates on the VERSION floor the code needs, not just
    shutil.which(tool) is not None
  • A gate that walks the REPO ROOT prunes .worktrees/ (another branch's checkout),
    or asks git ls-files instead of walking
  • A fixture that stubs away the only code path releasing a permit, lock or in-flight
    claim gives it back itself — restored to what the test INHERITED, not to a
    pristine value
  • A source ratchet strips docstrings with ast, not by subtracting __doc__ from
    inspect.getsource (3.13 dedents docstrings)
  • A test whose contract IS a real symlink is listed in test/requires-real-symlinks.txt;
    one that needs a directory that resolves elsewhere uses make_dir_link
  • No AsyncMock standing in for a synchronous method; every cancel() awaited. A
    bare AsyncMock() provider makes EVERY accessor an awaitable, and the sync ones
    (context_window_tokens(), mcp_session_report()) become never awaited warnings
    attributed to a LATER test — build the double from a factory that pins them as
    MagicMock, and a stand-in for a spawner or wait_for must close() the coroutine
    it swallows
  • No module-level asyncio primitive (Lock/Event/Future/in-flight dict) reachable
    from the code under test without a per-test reset
  • Nothing can block forever: every await the test itself must unblock is wrapped in a
    bounded wait_for; no sleep(0.05) standing in for "let the other task register"
  • A test of a coalescing / single-flight path holds the shared operation open (a
    gated stub) until every reader has registered, then releases it — an instant stub
    lets the leader finish before the others arrive, and a done future never yields
  • A module that rglob+ast.parses src/ once per module also carries
    pytestmark = pytest.mark.xdist_group(name="tree_scan_<module>"), or every xdist
    worker it touches re-runs the scan
  • A scan of the WHOLE repo goes through source_corpus.repo_files() /
    repo_files_named(...), never rglob/os.walk from the root — a walk descends
    gitignored trees and any nested worktree, so the gate reports that copy as the
    offender, or (with an any(...) assertion) keeps passing on it; the gate keeps its
    own scope filter, because _vendor is tracked
  • A fixture stamped from a module-level NOW is only compared by production code
    whose clock is pinned to that same NOW on the module-under-test's own time
    binding (D2) -- never two clocks
  • After await handler(...), an assertion on something a worker thread emits via
    call_soon_threadsafe waits on that signal, not on the handler returning
  • No upper bound on a rate, a sample count, or a measured duration beyond what D7
    allows
  • Source files read via _REPO_ROOT = Path(__file__).resolve().parents[N], never a
    relative Path("src/...") — xdist workers may change CWD
  • Passes at -n0 and under -n auto, and passes when run alone
  • No large allocation at module scope — especially not inside parametrize, where
    every worker pays it at collection and holds it for the session
  • Cross-platform: platform_compat for process/signal/lock calls, no assumption
    about path separators, case sensitivity, /tmp, or timer granularity
  • @pytest.mark.asyncio only on async def tests — never a module-level pytestmark
    over a file that also holds sync tests
  • Every aiohttp app[...] write happens before the TestClient/TestServer starts; an
    override of something a production on_startup hook creates is itself a later
    on_startup hook; an already-set-up runner's app is never wrapped in a second
    TestServer (serve runner.server through a ServerRunner)
  • A fixture venv is built with symlinks=not IS_WINDOWS, and every probe of the
    RUNNING interpreter's packaging (find_spec("pip")) is pinned — a uv venv has no
    pip, and a copied python-build-standalone binary does not start
  • A fake subprocess.run routes on whole argv tokens, never on a substring of the
    joined argv — a TMPDIR path element can contain any word, including the test's id
  • A directory chmodded to a non-writable mode under tmp_path gets owner rwx back in a
    finalizer; a nested pytest the test spawns gets --basetemp under the outer
    tmp_path (the default is the SHARED per-user tree every other run prunes)
  • A module- or session-scoped fixture that patches env does it through
    pytest.MonkeyPatch.context() scoped to what actually needs it, never a raw
    os.environ[...] write and never a MonkeyPatch held for the whole module when the
    value it builds is already materialised; an env value each test's subprocess must
    see (a git identity) is set per test through the function-scoped monkeypatch,
    since a session-lifetime patch still reaches every later suite on the worker
  • A default that is not HOME-derived (workspace_root() → /Volumes/workplace on
    macOS) is relocated by patching the resolver, and the result is asserted to be under
    tmp_path
  • A resolver that finds a per-user tool (gh, mise, say) is pinned at the seam
    production reads, never left to find the host's binary; an address probe that
    connects a datagram socket off-loopback is stubbed with an inert socket subclass
  • A stub for a runner that owns deferred cleanup (cleanup_paths) reaps them itself;
    a file with a .lock sidecar lives under tmp_path; a spawned real binary gets
    TMPDIR/TMP/TEMP inside the test's temp tree in its env
  • A thread count that rises is only a leak if the new threads are not a named bounded
    pool (mc-*, sel-writer) — print thread names in the probe first; a
    Thread(target=lambda: ...) that is expected to raise captures the exception and
    asserts on it instead of leaving a PytestUnhandledThreadExceptionWarning
  • A tree ratchet streams parsed trees and caches RESULTS, never a list of ASTs shared
    between scanners; a growth/linearity ratchet measures the algorithm's own work
    (bytes produced, items visited), not wall-clock and not interpreter call counts
    (str.join is one C call at any length)
  • Nothing assumes tmp_path is OUTSIDE a git repository: a fixture whose verdict comes
    from an upward walk (nearest .git, install.sh + setup.cfg, project markers from
    the cwd, git discovery) plants the boundary it reads — a developer's TMPDIR may sit
    inside the checkout, and in a linked worktree the .git such a walk finds is a FILE
  • A nested python -m pytest on files under tmp_path passes its own -c <ini>,
    --rootdir and --color=no: otherwise it adopts the repository's ini, and the
    repo addopts reshape the very output the outer test greps
  • A stub for an os.* function reached through a module alias
    (monkeypatch.setattr("<mod>.os.unlink", ...)) forwards every non-owned call to the
    saved real function with *args, **kwargs INTACT — dropping dir_fd re-aims pytest's
    own rmtree at the cwd — and asserts a bare relative name never reaches the stub
  • A test that compiles or imports a throwaway source under tmp_path keeps its bytecode
    there (sys.pycache_prefix under tmp_path, or the suite's no_bytecode() helper),
    because the suite-wide mirror is keyed on the absolute source path and a per-run path
    never gets read again
  • A fixture needing a SHORT path (AF_UNIX sun_path, a redaction-sensitive path) calls
    tmpdir_helpers.short_tmp_base() — never a literal /tmp, and never
    tempfile.gettempdir(), which is too long under a pinned TMPDIR
  • No raw os.kill(pid, 0) in test code — it TERMINATES the target on Windows; route
    liveness through platform_compat.pid_exists/pid_liveness, and any teardown signal
    to a pid published by a child captures the target's identity at spawn and revalidates
    before signalling
  • Every fetch seam a code path can take is routed, not just the one the happy path uses
    (a JSON + text stub still lets a BLOB fetch reach the network)
  • A skip condition is decidable the same way on every run of one host: no ctime tick, no
    "did a real tool answer in time", no "did an earlier test in this worker arm a hook" —
    build the condition, resolve it from disk, or measure in a fresh interpreter
  • A handle the object under test opens for the process lifetime (a store's SQLite
    connection + writer thread, a lazily-opened index) is closed by the fixture that built
    it; when production has no close path, that gap is the finding
  • No exception INSTANCE in a parametrize list (pytest.param(OSError(...))): the
    instance lives for the module, raise err hangs a __traceback__ on it, and the
    frames on that traceback keep every local alive — a handle, a lease, a socket — for
    the rest of the worker. Parametrize the errno and build the exception inside the test
  • A test that asserts process-global "nothing retained" state (lease._held, a
    registry, a pool) is only as good as the files that share the worker: the file that
    exercises the failure path pins the same table empty in its own teardown, so the
    retention is reported where it was created rather than forty files later
  • "Let it finish" is never a fixed sleep: wait on the state you are about to assert
    (poll it off-loop under a bounded deadline, or await its event) — two 200 ms sleeps
    that were enough at -n0 read starting for every row on a loaded Windows worker
  • An unclosed sqlite3.Connection is a self-cycle on 3.11+ and refcounting never frees
    its db/-wal/-shm. Close every inline VectorMemoryStore / KnowledgeStore /
    SkillsLoader / SubagentManager as the SQLite teardown item below says, e.g. through
    test/conftest.py's opened fixture. A KnowledgeStore's close() is fine on the
    owning thread; use _close_all_for_tests() when another thread held a connection
  • An exact bound on monotonic float arithmetic ((now + cap) - now <= cap) can miss by
    one ulp on some platforms; compare with a named slack (_FLOAT_SLACK = 1e-6, as in
    test/test_knowledge.py) or pytest.approx, never a bare <=
  • A fixture that plants a path under tmp_path and asserts a production "not under
    $HOME" refusal does NOT fire pins Path.home (the seam the product reads) to a
    sibling that is not an ancestor of the fixture — a developer's TMPDIR may sit inside
    home — and a mirror test plants INSIDE the patched home to prove the refusal still fires
  • A helper that asserts a WARNING count filters caplog.records by the logger the test
    enabled (rec.name == <logger>), never the unfiltered root capture: an executor thread
    another test's SessionManager armed logs on its own logger mid-test; and a test that
    builds a real SessionManager relies on conftest's _no_boot_sandbox_sweep pin rather
    than patching the sweep itself unless the sweep is its subject
  • A teardown that signals a pid the body has already proven dead (a reaped root, a
    grandchild init collected) signals only while process_start_time still matches the
    identity recorded at spawn, and refuses an unpinned pid; a "nonexistent pid" probe uses a
    number above /proc/sys/kernel/pid_max, not a convention
  • A resolver the product caches for the process (functools.lru_cached ssh -V,
    code_fingerprint()) is pinned at MODULE scope with an explicit opt-out fixture — pinning
    only the flagged tests moves the real spawn to the next reader; a class-local copy of a
    conftest fixture re-implements ALL of its pins or requests the original instead
  • A production git -C <scratch> spawn the test cannot pass cwd= to runs under a
    fixture monkeypatch.chdir(tmp_path), so nothing inherits the checkout as process cwd;
    the product keeps cwd=None there on purpose (a missing clone must stay git's rc=128,
    not a FileNotFoundError before git runs)
  • A backend a daemon reaps on disconnect goes through the pool's TRACKED reap
    (pool.spawn_shutdown), never an inline await shutdown() in a handler's finally:
    teardown cancels the handler after the backend has left the map, and the child then
    belongs to nobody
  • A child whose environment or argv the test must READ BACK through the kernel
    (KERN_PROCARGS2, /proc/<pid>/environ) is a non-platform binary -- the test's own
    sys.executable -- never sleep or env: macOS 26 answers an argv-only record for
    Apple platform binaries even to a same-uid reader, so the oracle reads None and a
    "refused" assertion passes for the wrong reason
  • The bounds on nested python -m pytest runs fit INSIDE the per-test --timeout: N
    sequential children each capped at 60 s inside a 120 s test can only fail the test
    ahead of pytest-timeout, never protect it; independent children run concurrently
    (wall = the slowest) and each gets the outer budget less a margin
  • A handle the product REWRITES asynchronously (a claim's .task that the run swaps for
    its inner task) is never read off shared state after an awaited response -- take it
    from the seam that hands it over (attach_run_task), or the assertion names whichever
    task won the race
  • A fixture never answers a production path with a FIXED absolute host location to keep
    a golden host-free: the product WRITES into the window it is handed (record_owner
    unlinks .owner, the gate probe shutil.rmtrees it), and a path that "never exists"
    is one mkdir on some host away from a real deletion -- answer a real directory under
    tmp_path and stub the golden's env contribution instead
  • A test whose red survives neither running the file ALONE nor running it WITHOUT
    -p probe_plugin is the probe's finding, not the suite's: a per-test observer that
    reaches os.path through the module a spied test patches records itself as the code
    under test
  • A child that polices its OWN peak memory reads /proc/self/status VmHWM on Linux,
    never getrusage(RUSAGE_SELF).ru_maxrss: execve seeds ru_maxrss with the parent's
    high-water mark, so a child of a 2 GiB xdist worker (or gateway) reads over its ceiling
    before it has parsed a byte -- green at -n0, red under the suite; the test that pins
    it plants a BLOATED PARENT and asserts the child still finishes
  • A test that plants a fake executable under tmp_path and expects a guard LATER than
    the working-tree fence to fire pins the fence root (_WORKTREE_ROOT) to a tmp_path
    sibling that is not the fake's ancestor -- under a repo-internal TMPDIR the fence
    wins first, and a patched which() that answers the fake for git too can hide it on CI
  • A module that boots the REAL start_dashboard pins
    hooks_integration._lifecycle_dispatcher / _route_registry to None BEFORE the boot
    (so monkeypatch restores them); a module whose assertions assume "never booted" says
    so with an autouse pin. Attribute a flaky 409 by its BODY (hooks disable failed: ... MagicMock ... await), not by the endpoint that answered
  • A PTY test that asserts a child receives a signal gates the send on
    os.tcgetpgrp on the PTY's own descriptor leaving the shell's group -- the line-discipline ECHO of the
    command is not proof the shell has read it (VINTR flushes unread input and the test
    passes with no child ever born) -- and reaps the session group in a finally; a spawn
    that hands a shell a terminal resets inherited SIG_IGN to SIG_DFL first, because a
    backgrounded launcher's ignored SIGINT survives exec into every descendant
  • A module-scoped autouse fixture never wraps the module in
    mock.patch.dict(os.environ, {...}): the keys are live between tests for every module
    the worker runs meanwhile, while the function-scoped conftest floor already overrides the
    home during each body -- set the feature flag per test with monkeypatch.setenv
  • A memoised per-file read on a tree scan (lru_cache on _read_text, a tuple(...)
    corpus helper) keeps every file resident for the process; the seam streams, and the test
    consumes the iterator (count, path set, zip(strict=True)) instead of materialising it
  • A test's own kiro-cli --version is never the HOST's: every agent-spec write reaches
    installed_kiro_cli_version(), cached per process, so the module pins it to
    SPEC_PERMISSIONS_MIN_VERSION (house fixture) or the host's install decides the
    contract under test
  • A coreutil a real-bash harness must neutralise (sleep in a retry backoff) is a shell
    FUNCTION defined at the top of the driver script, never an executable planted earlier on
    PATH: Git for Windows' bin\bash.exe launcher prepends /usr/bin to whatever PATH
    it is handed, so the shim never wins there and every failing case sleeps the whole
    budget; record each call and assert the SCHEDULE, not the clock
  • A Windows Job ActiveProcessLimit sized as "this child and nothing else" is
    1 + platform_compat.python_launcher_hops(): a venv's Scripts\python.exe is a
    redirector that spawns the real interpreter as its child, so a ceiling of exactly one
    refuses the spawn it was meant to bound (exit 101, Unable to create process using)
  • A test that asserts a key PASSES THROUGH a scrub (HOME, PATH) plants that key in the
    parent first — CI runners export HOME on Windows, a server session does not, and the
    assertion otherwise measures the host
  • A refusal raised by an object that holds a process-global resource (a crew-log handle
    and its write lease) is asserted through a CLASS-based context manager whose __exit__
    copies the exception's fields into a plain record and returns -- never
    pytest.raises(...) as exc (the ExceptionInfo and the test frame form a cycle that
    keeps append's self alive until the cyclic collector runs) and never a
    @contextlib.contextmanager (throwing into a generator adds the same cycle through the
    generator's frame); the file's autouse teardown pins the table (lease._held) empty
    without gc.collect()
  • A stand-in handed to a table that judges liveness by probe (RUNTIME_TENANCY._alive
    reads is_alive / is_process_alive) either answers the probe or its test releases the
    claim itself -- a fake with neither is alive for the rest of the worker -- and a stubbed
    _dispose_* still owes every release the real one performs
  • A test that hands the reaper a reset that never returns (_hanging_reset) pins
    _RESET_TIMEOUT the way its sibling classes do; a passing test whose wall time lands on a
    product constant (30.0 s, 60.0 s) with the CPU idle is spending that constant, not
    measuring it -- classify.py's TIMEOUT-SHAPED rows name them
  • monkeypatch.delenv(key, raising=False) on a key that is ABSENT records no undo: when the
    product then os.environ.setdefaults it, setenv the key first so the fixture's undo
    removes whatever the product leaves
  • A harness that runs a real shell script SUPPLIES every tool the script requires that
    the host may lack (jq beside the gh and curl it already stubs) as a shell
    function in its BASH_ENV file, implementing exactly the invocations the script
    makes and failing loudly on any other -- never a pytest.skip on the missing tool,
    which is a loosened ratchet (a-ratchet-may-only-tighten); probe the host through
    the bash the script will run under, not shutil.which, and prefer the real binary
    where that bash has one
  • A leaked child whose spawning thread is a NAMED pool reaper (mc-*-reaper) is the
    shared executors pool warming, not the test's child: the fix is at the pool (a
    shutdown that joins its reaper BEFORE the kill loop, so a tick past its stop check
    cannot refill the slot it just emptied) and at session end
    (shutdown_maintenance_executor() in a test/conftest.py fixture), never a reap in
    the first test that happened to trigger it
  • Two writers creating one sidecar on a fresh directory open it O_CREAT | O_EXCL first
    and reopen without O_CREAT on FileExistsError: a nonexclusive O_CREAT can lose
    the create race on Darwin with a bare ENOENT for the leaf, and a prune or flush
    that swallows it silently skips its work (_open_lock_sidecar)
  • A test double's pid (4242) is a shared name across hundreds of files, so any
    process-wide table keyed by pid (runtime_ownership's leases and tenancy) is reset on
    BOTH sides of every test by a test/conftest.py autouse fixture; a kill-gate assertion
    that reads refused where it expects the failed-kill wording is a neighbour's lease,
    not the gate
  • A race detector never parks on a threading.Barrier(..., timeout=) a correct
    interleaving cannot reach: the correct code then pays the whole timeout every run and
    pass is told from fail by elapsed time. Park on an Event, signal the arrival you are
    waiting for through the seam under test, and assert the event count while parked
  • monkeypatch.chdir does not put a cwd on the spawn: the descriptor still says
    cwd=None, indistinguishable to a per-spawn audit from a spawn in the checkout. Pin the
    helper's seam instead -- runner=partial(subprocess.run, cwd=...), or the script
    module's subprocess binding replaced with a namespace whose run carries cwd
  • An object whose constructor opens SQLite (SubagentManager -> tasks.db,
    KnowledgeStore per thread, SkillsLoader -> the skill index) is closed through its
    production close path at teardown -- test/conftest.py's close_subagent_managers /
    close_skills_loaders opt-in fixtures, opened(...), or _close_all_for_tests() when
    ANOTHER thread held a connection (close() is per-thread) -- never left to the cyclic
    collector, whose timing is what makes the descriptor count flap between runs