Build a reduced compiler by selecting frontends and targets
READ THIS FIRST — two things a reader will otherwise get wrong
1. Omitting frontends is NOT the size lever it sounds like. Measured
2026-08-19, same rig, fpc -O2:
| build | size | vs default |
|---|---|---|
| default | 3,376,848 | — |
| 9 frontends off | 3,228,480 | −4.4 % |
| + C frontend + 3 host backends | 2,677,120 | −20.7 % |
Nine frontends buy less than three host backends. The skeletons are a few hundred
lines each; pyparser.inc alone is 35,682 lines, an order of magnitude past every
other frontend combined. "Omit the languages you don't need" is the obvious lever and
it is the wrong one — the only frontend omission that moves size is PXX_NO_NILPY.
2. The NilPy-only configuration cannot be built today, at any size. It needs the
Pascal frontend out of the shared parser.inc, which is
[[refactor-a-carve-out-plexer-pparser-so-p-owns-its-own-files]] (Track A backlog,
p60) — already ranked work, not an open question. This ticket is blocked-by that
one for that configuration specifically; every other configuration is unblocked and
shipping. Do not quote a size for "the Python compiler for ESP at reduced size" until
the carve-out lands.
Thirteen omission defines ship today: PXX_NO_CFRONT _ZIG _BASIC _ADA
_LOLCODE _FORTRAN _ALGOL _ERLANG _WHITESPACE _RUST _I386 _ARM32
_AARCH64. (_RUST requires _ZIG + the six probe defines — see below.)
Filed 2026-08-19 at the user's request. The idea: pxx is now a big beast, but someone
who wants only a Pascal compiler should be able to build one. Frontend selection
(only-pascal, only-pascal-and-c, omit-nilpy) and target selection (only-esp-riscv)
compose, so only-pascal + only-esp-riscv produces a much smaller compiler. The umbrella
megalith stays the default; this is opt-in reduction, not a split.
The payoff the user named: "we get the 'python compiler for esp at reduced code size' almost for free." Names above are deliberately verbose placeholders — the switch spelling is still open, see below.
THE ACCEPTANCE TEST, from the user 2026-08-19 — and it settles the open question below
A reduced compiler — one frontend, one platform — must be able to build the FULL compiler from our source.
That is the whole test, and it is far stronger than "the reduced build passes its own tests".
It works because compiler.pas and every *.inc it pulls in — cparser.inc,
pyparser.inc, all five backends — are Pascal text. A Pascal-frontend-only compiler can
therefore consume the entire megalith source and emit the megalith binary:
reduced (pascal frontend, host target) --builds--> full compiler
full compiler --self--> byte-identical fixedpoint
Two properties fall out of one run. If a stripped compiler rebuilds the unstripped one, then the frontends and targets really are optional at build time — nothing outside them depended on their presence — and the Pascal frontend is complete enough to compile the whole project. A failure anywhere in that chain is a structural finding, which is the point: the user's stated reason is "to make sure we have a proper structure", explicitly not to generate more work for Track T.
This answers "what must a reduced compiler still self-host?"
Resolved, and more cleanly than the question was posed:
- The Pascal frontend is what makes a build a bootstrap candidate, because the compiler is written in Pascal. Any configuration including it can rebuild the full compiler.
- A configuration WITHOUT it — a C-only or NilPy-only build — is not a self-host candidate at all, and that is fine rather than a problem. It is a consumer artifact, gated on its own frontend's tests. The fixedpoint gate does not apply.
One precision the phrase "single frontend and platform" needs
The target must be the HOST platform for this test to mean anything. A Pascal-only build
restricted to esp-riscv can only emit esp-riscv binaries — it would produce an esp-riscv
"full compiler" that does not run here. True, and useless as a check. The structural test is
Pascal frontend + host target → full compiler → fixedpoint. Reductions to a cross target are
the product, not the test.
Why this is worth doing beyond the binary size
It is a falsifiable test of a design claim we make constantly.
devdocs/dev/the-substrate-is-ast-and-ir-not-the-parser.md says share the AST and IR,
duplicate the lexer/parser per language. If that has actually been honoured, omitting a
frontend is deleting some {$include} lines. Where it is NOT trivial, that is a
measurement of coupling we currently have no other way to take — and it points at real
design debt rather than at this ticket being hard.
MEASURED BEFORE FILING — the premise holds for C and fails for NilPy
Reference counts of frontend-specific identifiers inside the SHARED files:
| shared file | NilPy refs | C refs |
|---|---|---|
parser.inc |
909 | 6 |
ir.inc |
165 | — |
defs.inc |
96 | 4 |
symtab.inc |
85 | 3 |
ir_codegen.inc |
26 | — |
lexer.inc |
0 | 3 |
| total | ~1281 | 16 |
Read this honestly: these are counts of REFERENCES, not of edit sites. Many of the 909 will cluster into a handful of regions, and the number that actually need a guard is unknown until someone looks. It is an order-of-magnitude signal, not a work estimate — treat it the way this repo treats every "population where a problem is possible" figure.
What it does establish:
omit-cis close to the user's "trivial". 16 references across four files, and the C frontend is genuinely its ownclexer.inc/cparser.inc/cpreproc.inc(15,594 lines). R and Z, being greenfield, should be at least as clean.omit-nilpyis NOT.pyparser.incis 35,682 lines and carved out — but 909 NilPy references live in the sharedparser.inc, plus 165 inir.inc.lexer.inchas ZERO NilPy references, which is the useful contrast: the lexer was genuinely carved out topylexer.inc, and the parser was not.
This corrects a claim in CLAUDE.md
CLAUDE.md describes Track N as having "its own carved-out files" and contrasts it with Track
P, whose frontend "still lives inside the SHARED lexer.inc/parser.inc". Measured, that
is half right. N's lexer is carved out; N's parser is not — parser.inc carries 909 NilPy
references, more than P's own coupling is usually described as costing. This is very likely
why NilPy work keeps colliding with the A/P slot.
That finding is worth more than this ticket and should be checked before anyone plans frontend work on the strength of the current wording.
Shape of the work
- Precedent exists:
compiler.pasalready guards includes conditionally ({$ifdef PXX_NEED_FORWARDS}{$include forwards.inc}{$endif}), so the mechanism is in use, not new. - Targets look genuinely separable: five backend files —
ir_codegen386/_aarch64/_arm32/_riscv32/_xtensa— plus the sharedir_codegen.inc. Verify before assuming; a shared dispatch table keyed on target is the likely bear. - Suggested order, cheapest-proof-first: targets, then
omit-c, then R/Z, and NilPy last — by which point the 909 will have been characterised rather than guessed at.
Open questions (do NOT guess — file decide-* or ask)
Switch spelling and composition.ANSWERED (user, 2026-08-21, [[decide-reduced-compiler-switch-spelling]]): subtractivePXX_OMITis the ONLY mechanism — no positive list, no translation layer — plus a few sane prefab targets in the Makefile that expand to an omit list. The placeholder names (only-pascal,only-esp-riscv) are retired.- What a reduced compiler must still self-host. A Pascal-only build compiling
compiler.pasis coherent; a C-only build cannot self-host at all, since the compiler is written in Pascal. So the self-host gate is meaningful for some configurations and meaningless for others, and the gate must know which. This is the sharpest bear on the road and it is a design question, not an implementation one. - How the matrix is tested. Every configuration is a build that can rot silently. Track T will need an opinion, and testing all combinations is not affordable — a small set of named configurations probably is.
Gate
Track A's: make compiler/pascal26 (fixedpoint) + tools/gate.sh quick for the umbrella
build, which must stay byte-identical — a reduction feature that perturbs the default build
has failed. Each named reduced configuration additionally needs to build and pass its own
frontend's tests.
Plus the structural test above, which is the one that earns the ticket: a Pascal-only, host-target build produces the full compiler, and that binary reaches its own byte-identical fixedpoint.
And measure the second payoff rather than asserting it. The other motivation is a smaller and therefore faster compiler — "we only do Python, on ESP". Smaller text is a good reason to expect better instruction-cache behaviour and shorter dispatch, but faster is a claim about wall-clock, not about bytes. Report the size delta AND a compile-time measurement on a fixed workload. If size drops and time does not, that is worth knowing and does not invalidate the feature — code size is a legitimate goal on its own for an embedded toolchain.
Log
- 2026-08-19 — filed with the coupling measurement above.
MEASURED BY OMISSION (frank3, 2026-08-19) — the suggested order is inverted
The filing measurement counted references to frontend-specific identifiers and said so honestly ("an order-of-magnitude signal, not a work estimate"). It is worth replacing, because the reference count and the thing it stands in for disagree by two orders of magnitude in one direction and one in the other.
Method — the compiler is the oracle, not a grep. Comment out a component's {$include}
lines in compiler.pas, compile with FPC (-Se999, so it reports everything instead of
stopping at 50), count what breaks. A scratch copy of compiler/ plus lib/asmcore
compiles clean in 4 seconds, so the whole matrix is a couple of minutes. Nothing here is
inferred.
| omit | errors | files | where the coupling is |
|---|---|---|---|
| zig | 3 | 1 | compiler.pas |
| nilpy | 7 | 3 | compiler.pas:6, parser.inc:1, rtti_emit.inc:1 |
| arm32 | 8 | 3 | asmfront.inc:6, ir_codegen.inc:1 |
| i386 | 10 | 4 | asmfront.inc:6, ir_codegen_arm32.inc:2 |
| aarch64 | 10 | 3 | asmfront.inc:6, ir_codegen.inc:3 |
| cfront | 11 | 2 | compiler.pas:10, parser.inc:1 |
| rust | 200 | 7 | zparser.inc:123, gparser.inc:23, eparser.inc:23, fparser.inc:19 |
| xtensa | 288 | 5 | symtab.inc:179, parser.inc:62, exception_emit.inc:45 |
| riscv32 | 518 | 7 | symtab.inc:222, cparser.inc:123, exception_emit.inc:91, parser.inc:74 |
(Errors, not edit sites: one guard can silence several, and a few are cascades. Read it as an upper bound with a reliable ORDER.)
What this changes
1. NilPy is the CHEAPEST frontend to omit — 7 errors, fewer than C's 11. The ticket puts
it last, on the strength of 1281 references of which 909 are in parser.inc. Both figures are
real; they just do not measure separability. Those 909 are NilPy-aware behaviour inside the
shared parser (if isNilPy then ...), which stays compiled and inert when pyparser.inc is
gone. The compile-time surface is seven names:
PyLexAppend PyDfltPendFor PyDcEqProc PyDcReprProc PyExpandFStrings
PyLexAll ParsePyProgram
So the design claim in the-substrate-is-ast-and-ir-not-the-parser.md HOLDS for N, and
the correction the ticket makes to CLAUDE.md ("N's parser is not carved out") is itself half
right: N's parser file is carved out cleanly — what is not carved out is the shared parser's
knowledge of N, which is a different property and does not block reduction.
2. Targets are the bear, not the cheap first step. riscv32 (518) and xtensa (288) are
20-70x the cost of any frontend. The cause is not dispatch — it is that shared code emits
instructions inline: symtab.inc, exception_emit.inc and parser.inc reference
reg_t0 (81), reg_sp (62), rv32_sw (49), rv32_lw (38) and friends directly, and
cparser.inc carries 123 of them. x86-64 has no ir_codegen_x64.inc at all — it lives
inside the shared ir_codegen.inc, so the default target is not omittable by this
mechanism and would need a different one.
3. The flagship configuration dodges the expensive work entirely. The user's stated
payoff is "a python compiler for esp at reduced code size" = keep NilPy, keep xtensa and
riscv32, drop the rest. The costly omissions are exactly the two targets that configuration
KEEPS. only-nilpy + only-esp needs: omit cfront (11), rust (200 — see below), zig (3),
i386 (10), arm32 (8), aarch64 (10). Do the cheap six and the headline configuration exists.
4. R and Z share helpers, which is a genuine finding against the design rule. Omitting
rparser.inc breaks zparser.inc in 123 places, plus gparser/eparser/fparser — the
greenfield frontends call each other's support functions, which is the exact thing
the-substrate-is-ast-and-ir-not-the-parser.md says not to do ("duplicate the parser, the
lexer and their support functions per language"). Worth its own ticket; it makes R and Z
individually unomittable while costing nothing today.
5. One misplacement, cheap to fix. 6 of cfront's 11 errors are AddPasUnitDir /
AddPasIncDir — generic search-path functions that happen to live in cpreproc.inc. Moving
them to a shared file drops omit-c from 11 to ~4. Not coupling; filing.
Revised order of attack
omit-zig,omit-nilpy,omit-cfront(3, 7, 11) — cheapest, and they prove the mechanism end to end. MoveAddPasUnitDirout ofcpreproc.incfirst.omit-i386/omit-arm32/omit-aarch64(8-10 each, nearly all inasmfront.inc). After 1 and 2,only-nilpy + only-espis reachable.omit-rust— blocked on untangling R/Z's shared helpers; file separately.omit-riscv32/omit-xtensalast, and only if wanted: 800 errors between them, caused by inline instruction emission in shared files. This is the real design debt the ticket hoped to surface, and the flagship configuration does not need it.
Escalated, not guessed
[[decide-reduced-compiler-switch-spelling]] is filed and is now in decided/.
The second was not filed, 2026-08-30. This section read "Both open questions are filed
to Track U rather than settled here" and named
decide-what-a-reduced-compiler-must-still-self-host — which resolves to no ticket under
any name. So the sentence asserting that nothing here was guessed is itself the unchecked
claim: half of the escalation happened, and the citation covered for the half that did not.
Someone must still file it — it is the question of what the reduced configurations owe
the self-host gate, and it gates the acceptance test below. Left as an open name rather than
answered here, because guessing it is exactly what this section says it refused to do.
The acceptance test and the measurement above INTERACT — read them together
The structural test is Pascal frontend + host target. Against the omission table that is not the cheap corner of the matrix, and it is worth knowing before picking a first configuration:
- Frontend side: cheap. Pascal-only means omitting cfront (11), nilpy (7), zig (3) and
rust (200, blocked on R/Z's shared helpers) — plus the small frontends (
b,l,f,g,e,w,a), unmeasured but each a few hundred lines. - Target side: this is where the bear lives. "One platform" = host = x86-64, so it means
omitting riscv32 (518) and xtensa (288) — the two most entangled components in the
tree. The structural test therefore requires the expensive omissions, while the product
configuration (
only-nilpy + only-esp) keeps them. The cheap path does not reach the test.
So split the two rather than blocking one on the other:
- Frontend reduction first, all targets kept.
pascal-only, host build, still emitting every target. That is ~220 errors' worth of guards (200 of them rust's tangle) and it already proves most of the acceptance test's claim: a compiler with ONE frontend rebuilds the megalith, so nothing outside the frontends depended on their presence. Cheapest run at the real property. - Target reduction second, which is where
riscv32/xtensa's 800 inline-emission references have to be untangled, and which completes "one frontend, one platform". - The product (
only-nilpy + only-esp) needs neither of those two omissions and is reachable from step 1's mechanism alone.
Stating it because "one frontend, one platform" reads as a single step and is two, with the expensive half not on the path to the user's stated payoff.
The speed claim carries a measurement obligation
The second motivation is a smaller "and hence faster" compiler. Smaller text is a fair
reason to EXPECT better i-cache behaviour and shorter dispatch, but faster is a wall-clock
claim and bytes are not. Whoever lands a configuration reports both: the size delta and a
compile-time measurement on a fixed workload, on the same pin — a mid-session pin bump
steals the credit. (De-linked 2026-08-30: measure-before-and-after-on-the-same-pin was
never a ticket and no ticket on this board has ever used a measure- prefix. It is a
method rule, and the sentence states it in full without the brackets; the link only made
it read as an unmet dependency.) If size drops and time does not, say so — code size is a legitimate goal on its
own for an embedded toolchain, and an unmeasured speed claim is exactly the kind that gets
quoted back at us.
CORRECTION TO MY OWN MEASUREMENT — and the mechanism, landed for Zig and C
The omission counts above were TRUNCATED, and the NilPy conclusion drawn from them was wrong. FPC stops at the first fatal ("There were 7 errors compiling module, stopping"), so each run reported only the first layer and hid every later one — forward-declaration resolution in particular. I read a truncated list as a complete one and concluded NilPy was the cheapest frontend to omit. It is the most expensive.
This is the same failure the previous ticket in this lane was about: a mechanism reporting
on something adjacent to what was asked. grep counted references, which is not
separability, so I replaced it with the compiler — and then read the compiler's first
layer as its whole answer. The fix each round was to guard what it named, recompile, and see
what appeared underneath; four rounds, ~4 seconds each.
True compile-time surface, distinct names:
| omit | distinct names | verdict |
|---|---|---|
| zig | 0 | clean the moment the includes are guarded |
| cfront | 9 | CLexAppend CLexAll ParseCProgram CPreprocess AddDefaultCIncludeDirs ParseCUnit RegisterCMacroConsts CNodeDecaysToPointer FindCTypedef |
| nilpy | 183 | 177 forward declarations in parser.inc + 6 helper calls |
A forward declaration in parser.inc means the shared parser CALLS that body, so those 177
are real coupling, not bookkeeping. The ticket's original 909-reference figure was
directionally right about NilPy after all, and my first correction of it was wrong. The
useful part of my revision survives: the 909 are not 909 edit sites, they are 183 names —
one order of magnitude down, not two.
The target numbers (riscv32 518, xtensa 288) are from the same truncated runs and are therefore LOWER bounds. They stay last in the order for the same reason as before.
What landed
PXX_NO_ZIG and PXX_NO_CFRONT, both building clean, plus the mechanism the rest will use:
default 3,376,496 bytes (FPC seed build)
-dPXX_NO_ZIG 3,336,832
-dPXX_NO_CFRONT 3,153,568
-dPXX_NO_ZIG -dPXX_NO_CFRONT 3,112,496 -7.8%
compiler/frontend_stubs.inc holds one stub per shared-code entry point of an omitted
frontend, rather than a guard at each of its ~40 call sites: one place that records the fact
beats forty that re-derive it, and a guard forgotten at one site is a compile error in a
configuration nobody builds that day. Every stub raises rather than returning a plausible
value — each is unreachable by construction (its call site sits behind if isNilPy or the
C dispatch arm, and a compiler without the frontend refuses that source at dispatch), so a
stub that quietly returned 0 would convert "this build has no C frontend" into a wrong answer
far away.
AddPasUnitDir / AddPasIncDir moved out of cpreproc.inc into lexer.inc
([[refactor-a-search-path-helpers-live-in-the-c-preprocessor]], now done as part of this —
it was a blocker, not a cleanup: -Fu cannot be guarded out, it must work in a C-less build).
The default build is untouched: self-host fixedpoint converges and gate.sh quick is GREEN.
PXX_NO_NILPY was deliberately NOT landed. A half-implemented define that fails to
compile is a trap for whoever tries it next, so the partial guards were reverted rather than
shipped. Its 183 names are the next step and are mechanical but not trivial — each needs a
correct signature and a decision about what an unreachable body should do.
PROGRESS 2026-08-19 (frank3) — five omission defines, and the acceptance chain RUNS
Landed
| define | commit | what it removes |
|---|---|---|
PXX_NO_ZIG |
ecbcfd9b3 |
the Zig frontend |
PXX_NO_CFRONT |
ecbcfd9b3 |
the C frontend (+ frontend_stubs.inc) |
PXX_NO_I386 |
91ca417b3 |
the i386 backend |
PXX_NO_ARM32 |
ccef81c7c |
the arm32 backend |
PXX_NO_AARCH64 |
bde028cbe |
the aarch64 backend |
Every combination tried builds clean. The default build is unchanged throughout —
self-host fixedpoint converged on every increment, gate.sh quick GREEN, and all
five targets still emit and run.
THE USER'S ACCEPTANCE TEST RUNS CLEAN — with one honest caveat
FPC -O2 -dPXX_NO_ZIG -dPXX_NO_CFRONT
-dPXX_NO_I386 -dPXX_NO_ARM32 -dPXX_NO_AARCH64 -> reduced 2,789,936 B
reduced compiler/compiler.pas -> full1 (7.2 s)
full1 compiler/compiler.pas -> full2
cmp full1 full2 -> BYTE-IDENTICAL
cmp compiler/pascal26 full1 -> BYTE-IDENTICAL
Both properties the test was designed to catch hold: nothing outside the omitted frontends and backends depended on their presence, and the Pascal frontend is complete enough to compile the whole project. The stripped compiler's output is byte-identical to the repo's own self-hosted binary, not merely self-consistent — a stronger result than the chain required.
The caveat: this is not yet "one frontend and one platform". NilPy, Rust, Basic, Ada and Lua are still compiled in, as are riscv32 and xtensa. So the chain is verified for the reduction that EXISTS, and the acceptance test will have to be re-run at each further omission. It is not a one-time clearance.
THE SIZE/SPEED MEASUREMENT — size drops, speed does NOT
Recorded because the "smaller and hence faster" claim carries a measurement obligation, and the second half of it does not hold.
FPC -O2 seed builds, same tree, same flags:
| configuration | size | vs default |
|---|---|---|
| default (everything) | 3,376,608 | — |
-dPXX_NO_ARM32 |
3,309,872 | −2.0% |
-dPXX_NO_AARCH64 |
3,308,800 | −2.0% |
-dPXX_NO_I386 |
3,184,576 | −5.7% |
| three backends off | 3,055,792 | −9.5% |
| zig + cfront off | 3,112,608 | −7.8% |
| all five off | 2,789,936 | −17.4% |
Wall-clock, same box, hyperfine 40+ runs each, compiling test/hello.pas:
full 66.1 ms ± 4.5 [User: 45.9 ms]
reduced 81.2 ms ± 24.0 [User: 46.0 ms]
User CPU time is identical to within 0.1 ms. The wall-clock spread is this box's
noise (other agents are running; a repeated A/B with time reversed its own verdict
between rounds, and reduced's 163 ms max is a single outlier against a 61.8 ms min).
Compiling compiler.pas itself: 6.58 s full vs 6.77 s reduced, best-of-5 — same
answer, no gain.
This is the expected result once stated plainly, and it should be stated plainly
rather than left as an implication. The omitted code was never executed in the
full build either — it sits behind if TargetArch = ... arms that a host-target
compile never takes. Removing it removes bytes, not work. So the payoff of a
reduced build is footprint — binary size, resident memory, and the amount of code
shipped — not compile speed. Anyone repeating the "smaller hence faster" reasoning
should be pointed here.
WHAT THE DEFINES MEASURED — the coupling, which is half this ticket's point
Each omission forces the couplings into the open as compile errors. Four found so far, all real:
VariantTagForTk386— a target-INDEPENDENTTTypeKind→VT_*mapping living inir_codegen386.inc, called byir_codegen_arm32.incacross the backend boundary by that name. Moved toir.incasVariantTagForTk(91ca417b3).- The per-arch signal-runtime choice was inlined in the Pascal driver — five
if TargetArch = ...tests inparser.inc, three lines above the comment explaining that this exact shape is why "the other eight frontends shipped without" the I/O lock. Normalised intoEmitSignalRuntimeForTarget. The consequence — only the Pascal frontend emits a signal runtime at all — is a behaviour change and is filed separately as [[bug-a-only-the-pascal-driver-emits-the-signal-runtime]]. - The shared
-Opipeline names one backend's passes.ir_codegen.inccallsUnifiedResidencyAssignA64andFloatPoolBoundaryAssignA64unconditionally; both self-guard onTargetArchinside. Guarded, not moved. symtab.inc— the shared symbol table — carries three full function epilogues emitting raw machine code: i386 (inlineEmitB($0F)byte streams), arm32 (143 lines), aarch64 (173 lines), adjacent, eachExit-ing. Plusasmenc.incholds the per-arch inline-asm text routines for all five targets. This is the largest structural finding so far and it is the reason a "backend" is not two files: backend code lives in at least four shared files. Guarded here; a ticket for the shape follows.
A MEASUREMENT FAULT IN MY OWN RIG, and what it cost
compiler.pas carries {$UNITPATH ../lib/asmcore}, which FPC resolves relative to
the working directory. My scratch tree held a lib/ copied in an earlier session,
so every trial build linked a stale snapshot instead of the tree under test. It
produced working binaries and clean error lists, so nothing looked wrong. Found only
when a second script with a different directory layout failed outright.
The numbers above were all taken after the rig was fixed to symlink the real lib/.
Cost: nothing this time (lib/asmcore is untouched by any omission define), but the
same fault under a different change would have produced confident wrong results —
the second measurement fault in this ticket, after the truncated FPC error lists.
Both had the same shape: the rig answered a slightly different question than the
one asked, and answered it fluently.
Make that three, counting one that was not mine — a job count relayed between
sessions without being re-derived (six examples/tk jobs; there are seven). The
common factor is not carelessness, it is fluency: none of the three produced an
error, and all three produced plausible output. A rig that fails loudly costs
minutes. A rig that answers a neighbouring question costs a wrong conclusion written
into a ticket, which is the expensive failure this repo already has a playbook about
(devdocs/dev/root-cause-over-microfix.md, and the PXXDBG note in CLAUDE.md — both
exist because reasoning was cheaper than measuring and won).
Carried forward as an instruction, not an observation: the riscv32/xtensa
increment is where a stale-path fault would be hardest to spot, because ESP work
spans compiler/, lib/rtl/platform/esp/** and examples/esp32/** — three trees a
scratch rig could silently snapshot. Verify the rig against a KNOWN answer before
trusting a new number from it.
The findings this survey turned up before touching riscv32/xtensa
Filed rather than folded in, because neither blocks the omission work — they are
TargetArch comparisons and shared-file placement, not references to backend symbols:
- [[refactor-a-backend-machine-code-lives-in-six-shared-files]] — the structural one.
(Slug corrected 2026-08-30: this line still said
-four-, which resolves to nothing. The rename is recorded further down this same file — "renamed from-four-" — so the dead trail and its correction have been sitting forty lines apart in one document.) - [[refactor-a-nineteen-copies-of-does-this-target-link-the-builtin-unit]] — the
ESP-bare predicate written out nineteen times verbatim in
parser.inc.
REVISED ORDER OF ATTACK
— done.omit-i386/omit-arm32/omit-aarch64omit-riscv32/omit-xtensa— next. Bigger (800+ errors on the first truncated run, and that figure is a LOWER BOUND, see the truncation note above), because the ESP work put riscv32 and xtensa into the platform/PAL layer too.omit-rust— blocked on R/Z's shared parser helpers ([[refactor-a-the-greenfield-frontends-share-each-others-parser-helpers]]).omit-nilpy— 183 distinct names, 177 of them forward declarations inparser.inc. Deliberately unbuilt: a define that cannot compile is worse than no define. It is the measurement, not the goal.
Only after 2 and 4 is only-nilpy + only-esp — the configuration the user actually
asked for — reachable.
Make that four — and the fourth was caught, which is the point
The riscv32/xtensa survey opened with try.sh reporting exactly 60 errors for
PXX_NO_RISCV32, exactly 60 for PXX_NO_XTENSA, and exactly 60 for both together.
The script ended in | head -60. Three configurations agreeing to the digit is not a
finding, it is a pipe.
This is the same fault as the truncated FPC error lists, the {$UNITPATH} stale-lib/
snapshot, and the relayed job count — no error, plausible output — but it is the first
one caught before it reached a ticket, and it was caught by the shape of the number
rather than by a second tool disagreeing. The habit that did it is cheap enough to
state as a rule:
A measurement that comes out the same across configurations you expected to differ is a measurement of the harness. Check the harness before checking the theory.
The real figures, after removing the head: PXX_NO_RISCV32 518 errors (symtab 222,
cparser 123, exception_emit 91, parser 74, asmfront 6, ir_codegen 1); PXX_NO_XTENSA
288 (symtab 179, parser 62, exception_emit 45, ir_codegen 1); both, 805. The earlier
"800+ lower bound" was accurate — and was a bound on the pair, not on either alone.
Two files the earlier increments never touched fell out of this, and one of them is
another lane's: see
[[refactor-a-backend-machine-code-lives-in-six-shared-files]] (renamed from -four-).
cparser.inc — the C frontend — emits the C _start stub as raw riscv32 machine code,
so omitting riscv32 cannot be done without editing a Track C file. That, plus the fact
that riscv32 is dual-role (bare ESP32-C3 and hosted linux/qemu-user) while xtensa has
no hosted leg, means "omit the backend" and "omit the ESP platform" are not the same
switch. Both forks are escalated, not settled here; the second one lands on top of
[[decide-reduced-compiler-switch-spelling]].
Worth recording against the goal: this increment does not advance
only-nilpy + only-esp, the configuration actually asked for — that one keeps
riscv32 and xtensa. Its value is the inverse config (host-only) and the coupling it
exposed. The frontend axis, not the target axis, is the path to the stated goal.
The increment is PARKED, and this is why
PXX_NO_RISCV32 / PXX_NO_XTENSA are measured (above) and not landed. Not
abandoned, not blocked — parked, for a stated reason:
This increment serves the host-only configuration, which nobody has asked for.
Both configurations the owner described — a Pascal compiler that only targets ESP-riscv, and a reduced-size Python compiler for ESP — keep riscv32 and xtensa and drop the host backends. So 805 errors of coupling work here buys the inverse of the goal. The remaining distance to either config runs along the frontend axis (NilPy, Rust, Basic, Ada, Lua) plus eventually x86-64, and that is where the work goes next.
Unpark when someone wants a host-only compiler, or when the frontend axis is done and x86-64 omission makes the target axis live again. The measurement above is the whole cost of re-entry — do not re-measure it.
Pre-register the implausible result
Four measurement faults in one session (truncated FPC lists, the {$UNITPATH} stale
lib/, the relayed job count, head -60). The thing they share is not carelessness:
all four were silent successes. None errored. All produced plausible output. Three
of the four were caught only because something looked too round or too stable — that is
instinct, and instinct does not scale to the next agent or the next session.
The one discipline that catches these by method rather than by luck:
Before taking a measurement, decide what an implausible result would look like.
For an omission define: if two different backends produce the same error count, the
harness is lying. Written down before the run, 60 / 60 / 60 fails that test
instantly; discovered after, it took a second look to notice. Pre-registration costs one
sentence and converts a lucky catch into a repeatable one.
The frontend axis — eight defines landed (2026-08-19)
PXX_NO_BASIC, PXX_NO_ADA, PXX_NO_LOLCODE, PXX_NO_FORTRAN, PXX_NO_ALGOL,
PXX_NO_ERLANG, PXX_NO_WHITESPACE, PXX_NO_RUST — joining the five already in
(NO_ZIG, NO_CFRONT, NO_I386, NO_ARM32, NO_AARCH64). Thirteen.
There were more frontends than either the queue or I had counted. l is
LOLCODE, not Lua; and f/g/e/w are Fortran, ALGOL, Erlang and Whitespace —
six esoteric probes, not one. wparser.inc has no lexer at all (Whitespace reads
Source directly), so it needed a begin/end block built around its dispatch
arm before it could be guarded like the others.
Cost, measured, per frontend omitted alone
| define | errors alone | where |
|---|---|---|
| BASIC / LOLCODE / FORTRAN / ALGOL / ERLANG / WHITESPACE | 2 each | compiler.pas driver dispatch only |
| ADA | 3 | dispatch + one unsolved forward (below) |
| RUST | 198 | six other frontends' parsers |
The six probes costing exactly two symbols each is the good news and it is
structural, not luck: a frontend's whole coupling to the rest of the compiler is
its XLexAll / ParseXProgram pair in the driver, which is a dispatcher naming
its cases — the legitimate kind. Guarded with the existing PXX_NO_ZIG refusal
shape, so an omitted language is refused at dispatch with a message naming the
define, not by a mystery parse failure. Verified: all eight refuse correctly in a
reduced build, all eight still compile and RUN in the default build.
Two findings, both filed
AIntToStr was living in the Ada frontend. The A is for AnsiString, not
Ada. It is the compiler's own IntToStr, called ~40 times from the Pascal parser,
the NilPy parser, the C parser and the C preprocessor — with its body in
aparser.inc, a 460-line skeleton. Every one of those frontends silently depended
on Ada being compiled in. Moved verbatim to a new compiler/util.inc;
[[bug-a-aintostr-returns-empty-for-negative-numbers]] files the defect I found
while moving it and deliberately did not fix in the same change. util.inc sits
after lexer.inc, not before, because AppendChar — an equally general string
helper — is itself defined in lexer.inc. Same finding one level down, left alone.
Rust cannot be omitted alone. Six later frontends were built on rparser.inc's
helpers; see [[refactor-a-seven-frontends-borrow-rust-parser-helpers]], which
separates three layers wrongly wearing one R prefix — AST constructors (share,
wrong file), RWiden (numeric semantics, should not be shared: Zig has no
implicit widening yet calls Rust's), and REmitParamRegSpill (raw x86-64 inside a
frontend). PXX_NO_RUST therefore requires PXX_NO_ZIG plus the six probe
defines. Documented, not enforced by making one define imply the others —
an implicit expansion would hide precisely the coupling worth removing.
That is now the third frontend caught holding private target machinery
(Pascal: the signal runtime; C: the _start stub; Zig-via-Rust: register spill).
Three is the count this repo's own root-cause-over-microfix calls a design flaw
rather than a smell: there is a missing layer between the frontends and the
backends, and each frontend has independently grown its own copy of it.
Payoff, measured — and the frontend axis is the WEAKER one
Same rig, same fpc -O2, one run each:
| build | size | vs default |
|---|---|---|
| default | 3,376,848 | — |
| 9 frontends off | 3,228,480 | −4.4 % |
| + C frontend + 3 host backends | 2,677,120 | −20.7 % |
Stated plainly because it cuts against the redirect that brought me here:
omitting nine frontends buys 4.4 %, less than omitting three host backends.
The skeletons are small — a few hundred lines each; only Rust is 3,198. The weight
is elsewhere, and the numbers say where: pyparser.inc is 35,682 lines, an
order of magnitude past every other frontend combined. Any real size win on this
axis is PXX_NO_NILPY, and the symmetric one — a NilPy-only compiler — needs the
Pascal frontend out of the shared parser.inc, i.e. the long-deferred
plexer/pparser carve-out, not a define.
Acceptance test, re-run at 13 defines
Required at each increment, not once. Reduced compiler = 6,047,965 B (−21.2 % off the self-hosted 7,679,385):
pascal26 --(13 defines)--> reduced --> full1 --> full2
cmp full1 full2 BYTE-IDENTICAL
cmp pascal26 full1 BYTE-IDENTICAL
The reduced compiler builds the full compiler, and what it builds is bit-for-bit
the tree's own self-hosted binary — stronger than the test asks for. Gate:
make compiler/pascal26 converged in 1 round, gate.sh quick GREEN.
Pre-registration, used four times, paid twice
Following the rule banked above, each measurement got a written-down implausible result first.
- "Six frontends should not produce equal counts" → they produced 3, 3, 3, 3, 3, 3. Investigated, and it was genuine — two driver symbols each. The rule does not only catch faults; it also converts a suspicious-looking result into a checked one, which is how the "two symbols" finding got noticed at all.
- "
NO_RUST+NO_ZIGshould drop ~123 to about 77" → 74. Held. - "
NO_RUSTalone should still fail" → 198. Held. - "Size drop between 5 % and 10 %; a 0 % or a >40 % is the rig" → 4.4 %, just under the band. Re-checked the binary was the reduced one before believing it.
Zero rig faults this increment, after four in the previous ones. One process note
worth keeping: pxx's output path is a positional second argument, not -o.
Passing -o made it report ok: -o and write a file literally named ./-o in the
repo root — twice — while printing a success line with plausible code/data sizes.
A tool that succeeds at the wrong thing, again.
CORRECTION: every survey figure in this ticket is a FLOOR, not a cost
Found 2026-08-19 while surveying PXX_NO_NILPY, and it retroactively qualifies
numbers recorded above.
-dPXX_NO_NILPY reported 7 errors. Pre-registration said "hundreds, concentrated
in parser.inc's forwards; under 50 means the include was not guarded." The guard had
applied — so the prediction was what needed checking, and checking it is what found
this:
FPC does not run its end-of-unit unsolved-forward pass when the module already has errors.
Fatal: There were N errors compiling module, stoppingaborts first.
Silence the 7 identifier errors (guard the driver branch, stub the two shared call
sites) and recompile: 191 Forward declaration not solved appear. NilPy's real
cost is ~198 symbols, not 7 — a factor of 28.
The evidence is clean rather than inferred: -dPXX_NO_ADA produced 1 forward error
and 0 identifier errors, so the forward pass ran and reported. NilPy produced 7
identifier errors and 0 forward errors, so it did not. Same compiler, opposite
behaviour, and the discriminator is whether any other error exists.
What this qualifies
The thirteen landed defines are unaffected and remain fully verified. Each was driven to zero errors and then to a built binary — a binary that links has no unsolved forwards by construction, so nothing can be hiding behind an abort. The acceptance test and the byte-identical fixedpoint stand.
What is qualified is every survey figure, i.e. every count reported for a define that was measured but not landed. All of these had ≥1 identifier error, so all are lower bounds:
| reported above | actually |
|---|---|
PXX_NO_RISCV32 518 |
≥ 518 |
PXX_NO_XTENSA 288 |
≥ 288 |
| both, 805 | ≥ 805 |
PXX_NO_RUST alone, 198 |
≥ 198 |
PXX_NO_NILPY 7 |
198 (measured properly) |
The "800+ lower bound" language used for riscv32/xtensa turns out to have been accidentally correct — it was hedging about ESP's platform layer, not about this.
The pattern, fifth instance
Same species as the other four: no error, plausible output. FPC printed a
well-formed error list and an accurate There were 7 errors line. Seven is a
believable number for a frontend omission — it is exactly what the six esoteric probes
cost. Nothing looked wrong, and the only reason it was caught is that a written-down
prediction said "hundreds" and the result said seven, so one of the two had to be
wrong and it was cheap to find out which.
That is the pre-registration rule paying for itself a second time, and it is worth being precise about how: it did not flag a suspicious number. It flagged a disagreement between a number and a prior belief, which is a much sharper signal than any property of the number alone. Seven errors is not suspicious. Seven errors when you predicted hundreds is.
Standing correction to the method, for this ticket and any future omission work:
An error count is only a cost if the build reached the end. Drive to zero and link a binary, or report the number as a floor.
PXX_NO_NILPY — measured in full, deliberately NOT landed
The measurement, driven to the point where the next step would have been to write code:
| stage | result |
|---|---|
guard the two {$include} lines |
7 identifier errors — a floor, see the correction above |
| + guard the driver branch, stub the 2 shared call sites | 191 Forward declaration not solved (177 parser.inc, 14 forwards.inc) |
| + guard all 191 forwards | 426 identifier errors, across 176 distinct NilPy functions — 425 in parser.inc, 1 in cparser.inc |
Heaviest call sites, which are also the best carve targets:
23 PyParseBoolExpr
19 PyCallMeth1
12 PyAugBinTok
9 PyIsIdent
9 PyForceVariant
8 PyStoredName
8 PyParseSliceTail
8 PyMakeDynAttrSet
8 PyHoistPark
7 PyNodeIsComplex
6 PyNodeIsPyList
6 PyMakeDynAttrGet
Pre-registration, third use, and this time it missed: I predicted 50-150 identifier errors and got 426. Recording that rather than quietly moving on, because a rule that only gets cited when it wins is not a rule. What it still bought was the question — 426 was surprising enough to make me check the shape of the work before starting it, which is what produced the decision below.
Why it is parked, and this is a root-cause call rather than a size complaint
Landing this would mean: 191 {$ifndef} wrappers inserted into parser.inc's
forward block, plus 176 stubs in frontend_stubs.inc, each signature-identical to
its real body, each depending on the claim that its 426 call sites are unreachable in
a non-NilPy build.
There is already a ranked, proven campaign to delete this coupling instead:
- [[task-a-carve-nilpy-selectors-out-of-parser-inc]] — split 1, landed
(
ParseClassRecordSelectors→PyParseClassRecordSelectors, 0 regressions over a 513-file.npysweep, self-host byte-identical). - [[task-a-carve-nilpy-lvalue-parsing-out-of-parser-inc]] — split 2, backlog, p45, with the method written down and proven.
Those splits remove NilPy functions from parser.inc. Every one they move is a
forward that no longer needs guarding and a stub that never has to be written. So the
two approaches do not merely overlap — they run in opposite directions, and doing
mine first makes theirs harder: the carve tickets would then be editing a file with
191 preprocessor wrappers threaded through the exact region they need to split.
[[root-cause-over-microfix]] names this case directly: prefer the change that deletes cases over the one that adds them, and measure by tickets-closed-per-change. 176 stubs closes one ticket and complicates two. The carve campaign closes three.
So: PXX_NO_NILPY follows the carve campaign; it does not precede it. The
numbers above are its re-entry cost — nobody needs to re-measure. They also give the
campaign a progress metric it did not have: 176 symbols / 426 sites remaining in
parser.inc, and PXX_NO_NILPY compiling clean is the objective definition of
"done" for the whole carve.
That is the second time this feature has produced a measurement that is worth more than the define it was taken for — see the instrument argument in [[refactor-a-the-missing-layer-between-frontends-and-backends]].
Lock released — 2026-08-20 (hardware loss, not a decision about the work)
The session holding this ticket (frank3) ran on the workstation borg,
whose PSU failed on the morning of 2026-08-20 — audibly, taking a household
fuse with it. Borg is down for several days (a replacement PSU is being
borrowed), and its working tree died with it, so nothing of this ticket is in
flight anywhere.
Moved working/ → unfinished/ on the owner's instruction: "any ticket
claimed as 'working on' is now invalidated."
Track A in unfinished/ normally means a possibly half-applied compiler change.
It does not here. Dev work has moved to /home/neo/frank1 on plexus, a
clean clone of origin/master taken after the failure, so whatever frank3 pushed
is landed and green and nothing is partially applied in any live tree. Re-claim
this the ordinary way; just re-verify the notes below against master before
building on them, since they describe a tree that no longer exists.
wasm32 was added unguarded, deliberately — and its guard is not one line
Noted 2026-08-28 by the wasm lane, so this ticket does not later discover the
shape by tripping on it. The 7th target landed with no PXX_NO_WASM32,
matching riscv32 and xtensa (the two most recent additions) rather than the
guarded aarch64/i386/arm32.
That was a considered call, not an oversight, and the reason is the part worth
having: wasm's guard has a wider footprint than the other targets'. A
target guard elsewhere wraps one ir_codegen_*.inc include. wasm needs three
includes wrapped, plus a call site:
| include | why it is separate |
|---|---|
ir_codegen_wasm32.inc |
the backend, same as any target |
wasmenc.inc |
module model + binary writer — ~19.5MB of BSS |
asmtext_wasm.inc |
the WAT emitter |
compiler.pas's TARGET_WASM32 output arm |
calls writeWasm, which lives in wasmenc.inc |
A partial guard would be worse than none. Guarding only the backend leaves
wasmenc.inc's BSS behind, and that BSS is the single largest thing a reduced
build would want to drop from this target — so the half-measure keeps the cost
and adds the complexity. Either all four move together or none do.
Also relevant to this ticket's own framing: that ~19.5MB is a measured figure (the first draft was 66MB against the compiler's own 76MB, right-sized before landing), and BSS is zero-fill-on-demand — so a non-wasm build pays address space rather than resident memory. Whether that is worth a guard at all is a judgement this ticket is better placed to make than the wasm lane was.
UNPARK 2026-08-31 (frankA) — the carve campaign landed, and it did NOT finish the job
The park above says PXX_NO_NILPY follows the carve campaign. All three carve
tickets are now in done/: refactor-a-carve-out-plexer-pparser-so-p-owns-its-own-files
(which is why parser.inc no longer exists — it is the pasparser_*.inc set),
task-a-carve-nilpy-selectors-out-of-parser-inc, and
task-a-carve-nilpy-lvalue-parsing-out-of-parser-inc. The re-entry condition
fired and nobody was watching for it, which is the only reason this sat.
But the metric this ticket handed the campaign — "PXX_NO_NILPY compiling
clean is the objective definition of done" — is not met. Re-measured:
| symbols | sites | |
|---|---|---|
2026-08-19, parser.inc |
176 | 426 |
| 2026-08-31, after the carve | 134 | 279 |
−24% in symbols, −35% in sites. Real progress, not the finish line. The command, so nobody re-derives it — it is one line and ten seconds:
fpc -O2 -Tlinux -Px86_64 -Se1000 -dPXX_NO_NILPY -FU<tmp> -o<tmp>/pc compiler/compiler.pas
A MEASUREMENT THAT LIED, and it lied by being correct about something else
The first run of that command answered 7, not 279. Seven is a believable
number — it is the shape the six esoteric probes have (two each, driver dispatch
only), so it read as "the carve is finished". It was correct, about the wrong
question: at that point only the two {$include} lines were guarded, so
pyforwards.inc was still supplying ~190 Py* forward declarations. FPC
resolves a name against a forward and does not complain until the end of the
module, which it never reached — it aborted on the 7 symbols that had no
forward at all. So the instrument was reporting symbols with no declaration,
which is not symbols with no body, and the difference is 272 sites.
What caught it was this ticket's own rule, three sections up: decide what an implausible result would look like before you run it. 176 → 7 in a campaign whose three tickets each described a bounded move fails that test on sight. The fix is mechanical — guard the forwards too — and the guarded number is 279. [[the-name-is-not-the-thing]], instrument-shaped: nothing errored, it answered.
What LANDED here, and what it is not
Byte-identical in the default build (9eb39c00c650 with and without the diff,
two fresh make compiler/pascal26 runs), so this is structure, not behaviour:
pylexer.inc/pyparser.incincludes guarded.pyforwards.inc: thePy*forwards guarded in three spans, around the three islands that must survive — the two strays its own header names (VariantCastToTemp,PromoDemoteToInt64) and the two bodies it carries that the shared parser calls unconditionally (ParseArgExpr,PyIsStrBaseTk).ParseArgExprgets a no-NilPy body:PyExprModecannot be True, so the hook isParseExpr. Spelled out rather than left to dead-code elimination.- The driver's
isNilPyarm refuses by name, the shape the other twelve use. usesof a.pymodule (pasparser_proc.inc) refuses by name — the one place the Pascal parser reaches into NilPy for something that is not an expression hook.rtti_emit.inc'sProcParamDefaultSymarm, which only the NilPy header parser can reach.
PXX_NO_NILPY DOES NOT BUILD, and is deliberately NOT in the documented
thirteen. A note at the include site says so and carries the 279/134 figure,
because a define that is advertised and fails is worse than one that is absent.
Where the remaining 279 are — five routines, not a diffuse mess
| file | sites | routines |
|---|---|---|
pasparser_expr.inc |
191 | ParseFactorCore 98, ParseFactor 56, ParseSimpleExpr 17, ParseTerm 11, ParseExpr 9 |
pasparser_lval.inc |
68 | |
pasparser_stmt.inc |
13 | |
pasparser_call.inc |
5 | |
pasparser_name.inc |
1 | PyIsClassTypeExact |
cparser.inc |
1 | PyStoredName — the C frontend depending on NilPy |
ParseFactorCore already dispatches to PyParseFactorCore and Exits at
pasparser_expr.inc:521. The 98 that remain are all below that line, guarded
by isNilPy rather than PyExprMode — NilPy arms inside the shared argument
loops, not inside the shared expression parse. That is the actual shape of the
remaining work and it is why the existing hook did not close it.
Filed as refactor-a-carve-the-nilpy-arms-out-of-the-shared-pascal-argument-loops.
This ticket goes back to unfinished/ behind it, for the same reason as before
and with a smaller number attached.