← board

META: dev throughput — Track A+* / Track T integration

The observation that started it

We are not token-limited, we are wait-limited. Weekly token budget goes unspent because agents sit on test runs. Two 554s gate.sh quick runs in a single session on 2026-08-01, both GREEN, both finding nothing — roughly 20 minutes of an agent doing nothing, twice, for zero information.

Meanwhile the watcher box (12-core xeon) is otherwise idle.

Why "Track M" is NOT being created

The instinct to give this its own letter is right about the kind of work and wrong about the file lane. CLAUDE.md's rule: a new place code lives → new lane (rare, resisted); a new kind of work over existing files → tag. Dev tooling is tools/** + tstate/** — that is Track T's file lane already, and T's charter is literally "Tools & Testing", explicitly "a tool used for testing, not regression testing only". So this is Track T with a meta- slug (the same convention as meta-dialect-extensions-and-fpc-strict). No new letter.

Measured 2026-08-01 (dev box, idle)

step time
one self-compile 5.74 s
make compiler/pascal26 = build + byte-identical verify ~12 s
testmgr --tier quick (= the test-quick target) 2–14 s
make test-nilpy 554 s
gate.sh quick (quick + selfhost + test-nilpy) ~650 s

Two facts that fall out and correct earlier assumptions:

  1. The build IS the self-host fixedpoint. The $(COMPILER) rule compiles twice, cmps byte-identical, and refuses to install without convergence (fails after 4 rounds). You cannot skip it and you should not want to — it costs ~6s, and it means a compiler that cannot reproduce itself never leaves the working tree. The scariest failure mode of "never test locally" is already self-detecting.
  2. testmgr --tier quick is already the fast canary — 2–14s of dense torture tests. The 554s is gate.sh adding test-nilpy on top, not anything slow about the quick tier.

The model

Work items

Status 2026-08-02

Five of the six original work items are closed: publish-selfhost-red-immediately, agent-side-tstate-watch (shipped as twatch --follow), quick-canary-for-nilpy-and-c, quick-gate-must-be-quick… and decide-gate-line-convention. Open: pinning (60), bench preemption (55), and the unticketed core-utilisation check in the bullet above — which is a verification, not a change: read testmgr's own jobs=N cap=N scale=N header from a report before touching any config.

Done when

An agent can land a compiler fix in ~15s of local work, and hears about any regression it caused within minutes, without ever running a suite by hand.

Measured 2026-08-03 — the claim, tested

User observation that prompted this: "as far as I observe track T works as intended, no new issues. Track A and T align well... maybe numbers can prove it." They do. Rerun any time with tools/tstate_stats.py — the point of writing it down is that the claim stays falsifiable.

418 runs on xeon, 2026-07-31 .. 2026-08-03:

median p90 max
commit -> first verdict 2.8 min 3.4 min 7.5 min
commit -> full-tier verdict 8.8 min 10.2 min 12.5 min
commit -> opt verdict 12.7 min 15.9 min 17.2 min
full-tier run wall 5.8 min 6.6 min 9.6 min
native run wall 1.8 min 1.9 min 5.2 min

"Done when: ... hears about any regression it caused within minutes" — met. The first verdict on a sha lands in under 4 minutes nine times out of ten, and never took more than 7.5 in this window.

A/T alignment, quantified. The watcher's auto-filed regression tickets: 80 fixed, 7 rejected, 2 still open — 92% of resolved ones were real bugs. That is the number behind "they align well": T is not spending other agents' triage cycles on noise. (Two of those 7 rejections were the enrollment cascade and its stale seed, both of which now have guards — [[task-t-suppress-autoticket-until-host-baselined]], [[task-t-seed-from-stable-defeats-rebuild]] — so the ratio should improve, not merely hold.) Only 6% of runs report NEW-RED at all, so the steady state is quiet, which is what "no new issues" looks like from the other side.

Core utilisation — the unticketed work item above, now VERIFIED, not tuned. testmgr's own header on this box reads tier=full jobs=2040 cap=24 scale=1.00. cap=24 is nproc*2 on the 12-core xeon, and scale=1.00 says calibration finds the box at reference speed. Nothing is misconfigured; per the item's own instruction, nothing was touched.

One assumption corrected. The full tier is 5.8 min median, not the "~45 min" that [[feature-twatch-full-tier-coverage-age]] was written against (that figure came from the older box). It covers 42% of tested shas with a median 25.8 min gap — but the max gap was 7.8h, so preemption starvation is rarer than assumed and still real.

Status 2026-08-03

Open: [[feature-t-testmgr-owns-pinning-interruptible]] (60) and [[feature-t-bench-idle-must-be-preemptible]] (55). Both carry their own priority and neither is blocked on this ticket.

Prio 80 -> 40. The urgency this was rated for was the wait-limited premise — agents burning 20 minutes on 554s gate runs that found nothing. That premise is measurably gone: the dev loop is the ~12s build, breadth is offloaded, and the report comes back in under 3 minutes. The umbrella stays open until its two remaining children land (its own rule), but it should not head the ready queue while doing so — an 80 here also propagates down its dependency edges and inflates everything it links.