← board

2026-08-30 — moved working/unfinished/ by the coordinator, lock not released. frankA surveyed this rather than building it, found the ticket's premise unestablished, measured both routes, and filed [[decide-nilpy-runtime-tax-serialise-the-image-or-defer-the-bodies]] [U p60]. It then prototyped the deferred-bodies arm and refuted its own recommendation — the compile decomposes 59% routine bodies / 28% runtime declaration+interface / 13% floor, and the 28% band is exactly what makes the serialiser route unconditional and exactly what defer-bodies cannot see. The U ticket now recommends A-first.

Nothing is half-applied: no compiler file was touched and compiler/builtin/** was restored byte-exact by sha256 against pre-experiment copies. working/ is a live lock meaning an agent is actively on it, and frankA has moved to feature-port-rtl-over-libc [A p55]. Parked-pending-a-decision is unfinished/, which is what this is.

Unparks when the owner rules on the U ticket, not before. Re-read frankA's measured decomposition before starting either arm — and note face 182a: the serialiser's hazard is the TENSE, not the count. 176 parallel arrays is not the finding; that every FUTURE array must be added or the cache silently emits stale code is.

Cache the compiled NilPy runtime unit image

Third in the line: bug-a-every-nilpy-compile-pays-a-fixed-nine-second-cost (8.62s -> 4.06s, four string hotspots) -> perf-a-every-npy-compile-still-rebuilds-the-whole-nilpy-runtime (5.36s -> 3.06s, the allocator's idiv and the inlined variant blob) -> this.

Read the second one's RESOLVED section before starting. Its "what is LEFT" table is the current profile and it is a different shape from the one the earlier tickets describe; starting from the old picture will send you at the text assembler, which is now 0.6%.

Why this is now the only lever left on the fixed cost

Measured at 13e196cc8, real -O2 compiler, 44,064 samples over three runs of a zero-byte .npy:

share of in-.text
builtin runtime blob (heap alloc/free + ansistr retain/release) 35.5%
parse + lex (ParseFactorCore alone 9.4%) ~25%
IR lowering (IRLowerAST 5.0%) ~7%
symbol lookup (UNameMatch 2.7%, FindUClass 1.4%) ~5%
the text assembler ~0.6%

There is no hotspot left worth a microfix: the 35.5% blob share is allocation and refcount traffic generated by the parse and the tables it builds, and the release fast path is already down to test/je/dec/jne/ret. The remaining ~2.9s is the honest cost of parsing 24,460 lines and building the AST, IR, symbol, RTTI and fixup tables for them. To go below it you have to not do it.

ParseFactorCore's 9.4% is real and separately filed (perf-p-parsefactorcore-walks-a-92-arm-name-chain-per-factor, Track P), but even a perfect fix there leaves ~2.6s.

The design, and the one thing that makes it dangerous

Serialise the compiled result of pylib/pyeval — emitted code bytes, Procs, Syms, UCls, the fixup and RTTI tables — and load-and-relocate it instead of re-parsing. The compiler has no unit-image serialisation today and emission is fused with parsing into one global Code[] plus global tables, so this is a Track A project, not an afternoon.

The sharp edge is not the serialiser, it is the KEY. Miss one input — a define, the target, an -O level, --threadsafe, PXX_MANAGED_STRING, --strict-*, a -Fu root that shadows a unit — and the compiler silently emits stale code. That is precisely the "plausible wrong value far from the cause" class CLAUDE.md names as this repo's worst, and a stale unit image would produce it at the widest possible blast radius.

So the design constraint is an allowlist, never a blocklist (debugging-playbook.md): key on the full flag set plus a hash of every unit source actually read, and refuse the cache on any flag the keying code does not recognise by name. A new flag must then be added to the key deliberately; forgetting one costs a cache miss, not a wrong answer. Storing the compiler's own build sha in the key makes a compiler change invalidate every image for free.

A cheap first cut that proves the machinery without the risk: cache only the exact default configuration (no flags at all, host target, -O2), refuse everything else, and have --where print whether the image was used.

Repro / the numbers to beat

: > /tmp/empty.npy
printf 'begin end.\n' > /tmp/tiny.pas
time ./compiler/pascal26 /tmp/empty.npy /tmp/o     # ~3.0s, 1781 procs, 1,241,361B code
time ./compiler/pascal26 /tmp/tiny.pas  /tmp/o     # ~0.27s

Gate: make compiler/pascal26 byte-identical fixedpoint, plus — and this is the one that matters — a check that a flag change invalidates the image. Assert the invariant (same key => same bytes, different key => a miss), not the current timings.

What this is worth

719 NilPy jobs per full tier at ~2.7s of avoidable fixed cost each is ~1,940 CPU-seconds, and the same ~2.7s is on every NilPy user's hello-world. Ranked 60 rather than 85 because the two cheap causes are already gone: the tax was 5.4s when this campaign started and is 3.0s now, so the remaining prize is smaller and the remaining route is the expensive, dangerous one. Raise it if the matrix figure after 13e196cc8 says otherwise.


Survey before building, 2026-08-30 (frankA) — the serialiser's surface is 176+ parallel arrays, and that changes the risk

Claimed and surveyed before writing code, because this ticket says "a Track A project, not an afternoon" and the survey decides which project it is.

Baseline re-measured at HEAD eb3b0fd5c642, not reused

workload HEAD ticket's figure (13e196cc8)
zero-byte .npy 2.39 / 2.51 / 2.42 s 3.0 s
begin end. (Pascal) 0.21 / 0.21 / 0.21 s 0.27 s
emitted code, zero-byte .npy 1,253,550 B, 1859 procs 1,241,361 B

loadavg 2.76 (the ticket's numbers were taken at 8.15). So the fixed tax is ~2.2s, not 2.7-2.9s — the prize is real but smaller than the ticket says, and some of the gap is the box, not the compiler. Re-measure before quoting.

The finding: a unit image is 176+ parallel arrays, and this repo has a NAMED failure class for exactly that

The design says "serialise Code[], Procs, Syms, UCls, the fixup and RTTI tables". Counted, those are not five things. The capacity growers in symtab.inc resize:

grower arrays
EnsureProcCapacity 100
EnsureSymCapacity 44
EnsureUFieldCapacity 32
subtotal, per-entity tables 176

plus Code[], the string pool, RTTI and fixup tables. defs.inc declares 242 array of globals in total.

Every one of them must be written by the serialiser, and every array added later must be added to it, or the cache silently emits stale or incomplete code. That is not a hypothetical: symtab.inc:3932 names this failure class in so many words — "the 'one of six parallel arrays not written' class this file's SymTR comment names" — and records a measured instance of it, twelve symbols that carried an immediate pointee over depth 0 because nine of twenty-one write sites touched only part of the tuple.

So the ticket's stated sharp edge (the cache KEY) is real but is the smaller of the two. The key's hazard is bounded and has a complete solution — hash the whole normalised argv plus the compiler's build sha, and an unrecognised flag changes the hash by construction, no allowlist to forget. The serialiser's hazard is unbounded and has no by-construction solution: it is a 176-entry checklist that every future Track A commit can silently invalidate, at the widest blast radius in the compiler.

What makes it safe-able, if it is built

Not enumeration. Cold-vs-cached byte-identity over a CORPUS. A missed array is only observable if some compiled program's output depends on it, so the gate must be "compile with the cache cold and warm, diff the binaries" across many .npy programs, not one. The coverage instrument already exists: Track T's 719 NilPy jobs. That converts an unbounded enumeration problem into a measurable coverage one, and it is the only form of the gate I would trust.

A single-program same key => same bytes check — which is what the ticket's gate section currently implies — would pass a serialiser that forgot 170 of the 176.

A second route the ticket does not consider, and it needs a decision

The ticket asserts "nothing short of caching the compiled unit image will move it". That is not established. The cost is parsing 24,460 lines, and the alternative to persisting the result is not doing the parse: defer each routine body, record its token range, and parse only the bodies the user's program actually reaches.

Evidence it is idiomatic here rather than speculative: the generic path already has the primitive — GenericMethodBodyEnd finds a body's token extent and AppendTokenRangeToTemplateArena buffers it for later parsing (pasparser_generic.inc). What does not exist is any body-skipping in the normal unit path (grepped; no hits).

Trade-off, stated honestly: deferred bodies persist nothing, so they have no staleness class at all and no 176-entry checklist — but they only help if most bodies are unreachable, which is unmeasured, and they do not help the parse of interfaces. Serialisation helps unconditionally but carries the checklist forever.

That is a fork about what to build, not about how, so it is escalated rather than guessed: [[decide-nilpy-runtime-tax-serialise-the-image-or-defer-the-bodies]].

Nothing is half-applied. No compiler file touched.


REJECTED 2026-09-01 (frankA) — the blocker's rejection is this ticket's too

[[decide-nilpy-runtime-tax-serialise-the-image-or-defer-the-bodies]] was rejected by the owner on 2026-08-31 as "neither A nor B — the premise is wrong". That decide was this ticket's ONLY blocked-by, and it was raised out of this ticket, so the rejection lands on the premise both share: there is no NilPy-specific runtime tax. The compiler parses its own 235,854 lines at ~12,000 lines/sec and pylib+pyeval's 25,551 at ~11,600 — the same rate. What this ticket called a tax is general compiler throughput applied to 24,000 lines.

The owner also named the real successor: [[perf-a-the-compiler-parses-at-12k-lines-per-second-find-out-why]] (A, p50, in the ready queue). Anyone arriving here wanting the .npy compile faster should go there — it is the same seconds, attributed to the mechanism that actually owns them rather than to the frontend that noticed.

Rejected rather than re-prioritised, deliberately. It was blocked-by a rejected ticket, so it could never become ready — and CLAUDE.md is explicit that parking such a thing at a low prio "keeps it in the ranker forever at zero value". Dropping the edge to make it rankable would be worse: it would restore a p60 ticket whose premise a measurement has already refuted.

Flagged by progress.sh check and relayed by frankB, who left it alone because it was owned here. The measured decomposition in the body (59% routine bodies / 28% runtime declaration+interface / 13% floor) is still good data and is why the rejection was cheap — it is kept for whoever takes the successor.