← board

A target's timeout class is decided by a substring, and test-nilpy's is right by accident

How it was found

frankA fixed a NilPy range() hang (15afe4effd79) and asked a good question: "worth checking whether testmgr's nilpy jobs have a per-job timeout at all — if they do not, this test could wedge a tier instead of failing it."

Measured rather than reasoned — classify() run over the actual make -n test-nilpy recipe, 3268 lines:

class: corpus    timeout: 1200s

So the answer to frankA's question is no, it cannot wedge a tier — a hang is killed and published RED at 20 minutes. That half is fine and needs no work.

The problem is why it lands there. classify() (tools/testmgr.py) is a substring match over the recipe text, and test-nilpy hits the corpus arm because its recipe contains sqlite (from test_nilpy_sqlite_crud.npy), plus lua and uforth. Nothing about NilPy, its size, or its runtime put it there.

Why this is a bug and not a curiosity

The trigger is a deletion anyone can perform while doing something else. Remove or rename test_nilpy_sqlite_crud.npy — a perfectly ordinary act — and test-nilpy silently reclassifies from corpus/1200s to unit/90s. Every slow-but-passing nilpy run then publishes a false RED, and nothing in the tree connects the deletion to the consequence. The person doing the deleting will be working on NilPy tests, not on the harness.

This is a repeat, not a hypothesis. testmgr's own source records uforth falling through this exact hole:

"uforth is a corpus like the others, it just lives outside library_candidates ... so the path heuristic missed it and it fell through to unit — a 90s timeout. That was survivable at 46s; enrolling Gerry Jackson's 13 ANS word sets took the job past TEN MINUTES, so testmgr killed it and published a RED that was purely the harness misjudging the class."

And the file already knows the general shape, in the selfhost arm:

"the job classes selfhost anyway, because make -n expands the $(COMPILER) prerequisite and THAT text contains compiler.pas. So the class is currently correct by accident of the prerequisite, not by anything about the script."

That arm was then made correct on purpose. test-nilpy has had no such pass.

The retry interaction, which frankA raised and which doubles the cost

corpus is in RUN_RETRY_CLASSES, so a job that genuinely hangs is retried RUN_RETRY_TRIES = 3 times: 3 x 1200 = 60 minutes of a run's wall clock before the failure is final. frankA's range hang would have done exactly that had it been in a wired test. Retry is right for the classes it was designed for (runtime-nondeterministic: qemu sockets, threaded sqlite, sharded conformance) — test-nilpy is in that set for the same accidental reason it is in corpus at all.

The fix, and explicitly not a checker

frankA's proposal, and it is the right shape: an explicit per-target class map for the handful of targets whose class actually matters, with the existing heuristic as the fallback for everything else. Roughly a dozen entries. A target in the map is immune to recipe drift; everything else keeps today's behaviour.

Not a checker. Same argument as audit-a-a-comment-asserting-an-invariant-is-a-claim-about-a-sibling-arm-nobody-checked: a heuristic that flags "this class may be accidental" would fire on most of the matrix and get scrolled past. The map is cheap, total for the cases that matter, and self-documenting.

While in there, decide test-nilpy's retry membership deliberately rather than inheriting it from corpus — it is a deterministic build-and-run suite, and RUN_RETRY_CLASSES' own comment says deterministic classes stay single-shot.

Provenance note

frankA declined to file this itself ("your call and your lane; I am not filing into T") — correct: T owns the tool. Filed by the coordinator under its tooling remit. The argument for filing despite the class being currently right is frankA's and is worth preserving verbatim: "currently right is exactly the reason to file it rather than not" — a correct-by-accident invariant with an easy trigger and no written reason is the configuration that breaks silently.

Gate

Track T's own, per its lane rules. Verify with short tiers and a scratch bare repo; do not exercise this with long sweeps.

Deprioritised 2026-09-02 — the Track T tooling backlog was cut as a pile

This ticket is not being called wrong. It was moved as part of a pile, not judged individually, and nothing here disputes its finding.

Owner decision. 73 of the 74 open track: T tickets were filed between 2026-08-31 and 2026-09-02, 58 on one day. The pile was too large to work through and returned almost nothing, and a ticket nobody will fix does not sit neutrally — it stays in the ranker forever at zero value, which is the argument CLAUDE.md already makes for a terminal folder over a low prio.

Four were kept in the ranker on a purely structural test — an active umbrella or a hard blocked-by: edge from live work: umbrella-one-full-tier-run-with-no-red-tier, feature-t-freebsd-image-and-runner, and the two regression-test-core-* reds that block the umbrella.

Kept, not deleted, for two reasons: so the finding is not rediscovered and refiled from scratch by the next agent who trips over it, and so it can be pulled back if what it touches becomes load-bearing.

To revive it: move it to the owning lane's backlog, set status: backlog, and say in the ticket WHAT CHANGED to make it matter now. Restoring it because it reads well is how the pile comes back.