A corpus tree's absence should be counted, not just echoed
Three library_candidates dependencies live inside test-core, and until
2026-09-01 they handled absence three different ways:
| row | tree | on absence |
|---|---|---|
test-fgl |
library_candidates/fpc-rtl |
guarded, echoes SKIP |
stb_sprintf_probe |
library_candidates/stb |
guarded, echoes SKIP |
crtl_tiny_regex_match |
library_candidates/tiny-regex-c |
hard error |
The third was fixed to match the first two, because 844 dark rows is worse than one announced absence. That fix is not the end of the problem, it is the other half of it.
Both failure modes have already happened here
- Unguarded: one missing checkout ends the run. 48% of
test-corewas not failing — it was never reached, andmake's own error line pointed atMakefile:9272while the command that actually failed was at 9537. - Guarded:
tools/testmgr.py'sTIERScomment recordstest-fglguarded on/usr/share/fpcsrc, absent from every box here, printingSKIP (no fpcsrc)and passing for its entire life without running once.
So neither "make it loud" nor "make it skip" is the answer on its own. A skip is only safe when somebody can see how many there were.
The mechanism already exists in this tree
test-c-abi-mixed-link prints N of M targets measured, K skipped and goes RED
when N is zero — a skip is free, but a run that measured nothing can never
report PASS. test-core has ~1745 rows and no equivalent: it cannot say how
many it ran, so a box with three missing trees and a box with none produce the
same green.
Suggested shape, deliberately not prescriptive about the implementation:
- every guarded row increments a skip counter with its name;
- the target prints
test-core: R rows ran, S skippedat the end; testmgrrecordsSin the run's JSON beside the existingskips/skip_holesfields, which already exist for exactly this purpose at the JOB level and would then mean the same thing one level down.
Why this is filed rather than done
The counting belongs in the harness, the rows belong to whoever owns each test, and getting it wrong in the safe-looking direction produces a number that always reads zero. Track T owns the tool. Filed by Track A/C after tripping over the unguarded row while gating an unrelated C-frontend fix; the guard landed in the same commit, the counting did not.
The recovery, measured after the guard landed
Same box, same tree, the only change being the guard:
| compile rows executed | make exit |
MISMATCH | |
|---|---|---|---|
| unguarded | 901 | 2 | 0 |
| guarded | 1744 | 0 | 0 |
843 rows recovered, and 0 MISMATCH in both runs — which is the whole
point. The unguarded run was not red because anything was wrong; it was red
because it stopped, and its zero mismatches covered barely half of what the
green run's zero covers. Two identical-looking zeros, one of them worth twice
the other, and nothing in either log distinguishes them.
That is the number this ticket wants surfaced. test-core currently cannot say
whether its green is a 1744-row green or a 901-row one.
Deprioritised 2026-09-02 — the Track T tooling backlog was cut as a pile
This ticket is not being called wrong. It was moved as part of a pile, not judged individually, and nothing here disputes its finding.
Owner decision. 73 of the 74 open track: T tickets were filed between
2026-08-31 and 2026-09-02, 58 on one day. The pile was too large to work through
and returned almost nothing, and a ticket nobody will fix does not sit neutrally
— it stays in the ranker forever at zero value, which is the argument CLAUDE.md
already makes for a terminal folder over a low prio.
Four were kept in the ranker on a purely structural test — an active umbrella or
a hard blocked-by: edge from live work:
umbrella-one-full-tier-run-with-no-red-tier,
feature-t-freebsd-image-and-runner, and the two regression-test-core-* reds
that block the umbrella.
Kept, not deleted, for two reasons: so the finding is not rediscovered and refiled from scratch by the next agent who trips over it, and so it can be pulled back if what it touches becomes load-bearing.
To revive it: move it to the owning lane's backlog, set status: backlog,
and say in the ticket WHAT CHANGED to make it matter now. Restoring it because it
reads well is how the pile comes back.