← board

xtensa should not need a flag to build a large image

Why it is prio 35 — raised to 50 and put back, with the reason recorded

I raised this to 50 and it was half wrong. Corrected by frank-coordinator the same day, who had endorsed it and then went and measured.

The original reasoning was: "Every xtensa program that is not the compiler itself fits inside CALL0's range, so the population that needs this is one program, and that program has an answer."

The half I claimed was false, and is not. I offered the xt_backjump row as a second member of the population — a 118 KB program, not the compiler, that stopped fitting. But that row is generated by awk in the Makefile, 3000 statements emitted specifically so the image crosses J's ±128 KiB, and the comment above it says so outright: "GENERATED: the source has to be ~130 KB." That is a population the test defines, not one it samples. The other ~170 xtensa rows are nowhere near the wall. Counting it is the same error as counting a build artefact in compiler/ as a source site: ask what the corpus is before you count what is in it.

The half that stands, and is the reason this ticket is worth keeping sharp: the margin is not a property of the program. 4419e1aa7 pushed that arm from building to refusing with a zero-byte change — 622444B of code before and after — by moving __pxx_run_finalizers to the tail while its earliest caller stayed at 59154. Nothing size-shaped could have predicted it.

So the honest statement is conditional: if a real xtensa image ever approaches 512 KiB, any unrelated RTL edit can break its build without adding a byte, and the diff will look nothing like the cause. Today nothing real is close. That is a genuine latent hazard and it is still not p50 work, because the failure is a hard build refusal that names its own workaround rather than a miscompile. Back to 35, with the mechanism written down as the reason it will matter when it does bite.

Do not re-raise this on an ESP-growth argument (frankS, same day): empty bare-profile xtensa is 43428 B and uses softfloat adds ~54 KB, so bare ESP has 5-12x headroom and several doublings to go. If anything re-ranks this, it is hosted xtensa — which grows deliberately as frontends lean on it as a differential oracle — and the signal to watch is hosted sweeps hitting this repeatedly, not a single synthetic row.

How that was measured

Direct swap of the one buildable file in the watcher's range, no bisect and no compiler rebuild -- builtinheap.pas is a builtin, consumed when compiling the target program, so the arms differ by that file alone:

git checkout 156be41b504a -- compiler/builtin/builtinheap.pas   # last good
  ./pascal26 --target=xtensa --platform=posix --xtensa-soft-mulhigh xt_backjump.pas
  -> ok:  [code=622444B  procs=171]
git checkout HEAD -- compiler/builtin/builtinheap.pas
  -> error: forward call to __pxx_run_finalizers at 59154 cannot reach 620060

Both arms ran and answered differently, so the comparison is not vacuous. The --xtensa-long-calls build of the same source succeeds at code=622444B, and the windowed arm builds with NO flag at 556908B -- the two ABIs lay the image out differently and only call0 is over.

Size is NOT the condition — distance is

Sharpened by frankS, 2026-09-01, from the numbers above, and it inverts the obvious reading:

ABI code size verdict
call0 622444 B FAILS
windowed 556908 B builds

Both are over CALL8's 524288. The larger image is the one that builds. So image size is not what decides this; max caller->callee distance is, and size is only a proxy for it. That is precisely why a commit which changed no bytes could flip it: 4419e1aa7 moved the callee, not the byte count.

Two consequences worth carrying:

Also from frankS, against my own argument: do not justify this prio with an "ESP images are trending large" story. Empty bare-profile xtensa is 43428 B and uses softfloat adds ~54 KB (see [[bug-a-the-esp32-bare-image-doubled-in-code-and-grew-half-again-in-bss]]), so bare ESP has 5-12x headroom and several doublings to go. The population that actually hits this is hosted xtensa, which is growing deliberately as frontends lean on it as a differential oracle — every new program compiled for it is another chance to land a RED whose diff looks nothing like a size change. That, not ESP, is the cost curve that justifies 50. Revisit upward if hosted xtensa sweeps start hitting it repeatedly.

What this does NOT change

The two candidates below are unaffected; nothing here argues for one over the other. It raises how often the default bites, not how it should be fixed.

2026-09-04 — the population is no longer hypothetical (frankA)

The first C translation unit ever built for xtensa ([[bug-a-no-c-translation-unit-builds-for-xtensa-and-two-different-things-stop-it]], closed the same day) hits this on its first try:

$ ./compiler/pascal26 --target=xtensa --platform=posix \
      test/c_crtl_syscall_guarded_bodies.c out
pascal26:58: error: target xtensa: the forward call to __pxx_run_finalizers at
  code offset 58526 cannot reach its body at 664880 (CALL0/CALL8 reach +-512 KiB)

It is not a large program. crtl is what makes the image large, and crtl is in every C program, so the conditional the p35 rested on is now satisfied by the whole C-on-xtensa cell rather than by one generated row. The error names the flag and the flag works, so nothing is broken and this is still a default problem — but it is a default that now fires for everyone rather than for a test harness.

Note the veneer idea in the header gets more attractive from this run, not less: the failing call is __pxx_run_finalizers, the same tail-placed RTL routine the 2026-08-31 note names, reached from a call site 606 KB earlier. A trampoline at the end of the calling body is well inside CALL0 reach.

I have not changed prio:.

The population is wider than "linking crtl": plain Pascal hits it too

Measured 2026-09-05 (frankS). The body above records this as "a property of LINKING CRTL AT ALL, so it is now the first thing every C program on this target meets". That is still true and it is not the boundary. No C and no crtl required — this is the whole program:

program ex;
uses sysutils;
begin
  try
    raise Exception.Create('boom');
  except
    on E: Exception do writeln('caught ', E.Message);
  end;
  writeln('done');
end.



--target=xtensa --platform=posix --xtensa-soft-mulhigh -Fulib/rtl
error: the forward call to __pxx_run_finalizers at code offset 58938
       cannot reach its body at ...

Add --xtensa-long-calls and it compiles (code=655212B) and runs under qemu-xtensa: caught boom / done, byte-identical to the riscv32 control.

So the property is not C, and not crtl: it is linking anything substantial. uses sysutils plus one try/except is as ordinary as Pascal on this target gets, and it is over the wall. Every hosted-xtensa measurement anyone reports carries the flag, for Pascal as much as for C.

That also makes this ticket the thing two p45s sit on top of — both now entered as blocked-by, so effective_prio carries 45 here without anyone re-ranking:

The offset in the message (58938 here, 58526 in the C case) is the call site, which barely moves; what moves is where __pxx_run_finalizers lands. That is the ticket's own point about the margin not being a property of the program.

Fixed for the case every real program meets, 2026-09-05 (frankS)

The wall was not really about image size — it was about ONE callee whose body is emitted last. EmitProgramEpilogue emits the __pxx_run_finalizers body after the main body, while every exit path in the program emits a FORWARD call to it, including the earliest. Its displacement is therefore (end of program - call site), so the first Halt site in any image over ~512 KiB of code cannot reach it with a 3-byte CALLn.

That explains the three things this ticket had recorded as separate puzzles: why the wall reads as "programs over 512 KiB" for every program shape; why an unrelated RTL edit (4419e1aa7) flipped an image without changing its size, by reordering that one body to the tail; and why all three known failures name the same proc — the C crtl case at offset 58526, #include <stdio.h> under posix, and the Pascal uses sysutils case at 58938.

Fix: EmitCallProc's xtensa arm now reserves the long form for a forward call to FiniRunnerProc unconditionally, XtensaLongCalls or not. This is not a size heuristic — it is the one callee we know by construction lands last. That is this ticket's own stated goal, "widen only the forward calls that need it", and it costs the wide slot at exit sites only: two call sites, and 4096 bytes less than --xtensa-long-calls on the sysutils program (651116 vs 655212). On a small program the two are byte-identical.

Verified by RUNNING under qemu-xtensa, not by link success. All three formerly failing cases now build with no flag: the Pascal one prints caught boom / done, and test/c_crtl_syscall_guarded_bodies.c — the case that refuted this ticket's original premise — runs and prints its real output. Regressions checked: altstack, heap, signal predicate all still run correctly on xtensa; the --xtensa-long-calls path is untouched and still works; x86-64, riscv32, arm32 and aarch64 all unaffected (the change is inside the xtensa arm).

test_xtensa_finalizer_call_reach wires it, deliberately without the flag, and compares against the same program built natively so the row carries no per-target constant.

What is NOT fixed — the ticket stays open

A forward call between two ordinary procs more than 512 KiB apart still refuses and still names --xtensa-long-calls. The general answer is unchanged and is still the untried one: whole-program relaxation, i.e. a second emit pass, since a call's fixups are whole-program where a jump's are per-body. A veneer pool does not rescue the general case the way this ticket hoped, and that is worth recording as a measured negative: at ApplyCallFixups every body is already placed, so a trampoline can only go at the end of the code section — and in the observed failure the call site was 586142 bytes from there, itself out of CALL0 range. The veneer has to be reserved during emission to be reachable, which is the same information problem as the call it would fix.

So this fix removes the case every real program meets first, and leaves the class. Whoever takes the general fix should know the veneer idea was measured and does not close it.

2026-09-05 (frankS, owner) — the prio decision, and both prior arguments are stale

The ticket deferred its prio to me explicitly. Making it, on measurement taken today rather than on either recorded argument, because both are now out of date and one of them was refuted by my own fix.

The 2026-09-04 refutation no longer reproduces

It said: the wall "is not a property of deliberately-large generated programs, it is a property of LINKING CRTL AT ALL", so every C program meets it. Measured today, compiler 5783500470d0, no --xtensa-long-calls:

program code result
#include <stdio.h> hello 651116 B links, and runs (hi, exit 0)
test/c_crtl_syscall_guarded_bodies.c — the cited program 675692 B links

Both are well over the 512 KiB wall and both build. The cited failure was "the forward call to __pxx_run_finalizers at 58526 cannot reach its body at 664880" — and f49c0e11f (this session) reserves the wide form unconditionally for FiniRunnerProc, which is that exact callee. So the refutation was true when measured and was made obsolete by the fix that landed after it.

This is the third time this session that a recorded reason outlived its truth, and the first where the thing that invalidated it was mine. Worth saying plainly: a premise-refutation is a dated measurement exactly like the premise it refutes, and it goes stale the same way.

What actually still refuses, measured

pascal26 --target=xtensa --platform=posix --xtensa-soft-mulhigh compiler/compiler.pas:

error: target xtensa: the forward call to CmpBits$18392 at code offset 370895
cannot reach its body at 23898992 (CALL0/CALL8 reach +-512 KiB)

This is the residual my own fix named and declined to cover — "a forward call between two ordinary procs more than 512 KiB apart still refuses". It is not FiniRunnerProc; the two are 23 MB apart. So the remaining population is real and is a population of one: the compiler itself.

The call: HELD AT 35

Not raised. The argument for raising was "every C program hits this", and that sentence is no longer true — I removed the case every real program met first.

Not lowered either, and this is the part I want on the record so nobody low-prios it later on the "population of one" reading. That one program is the one the goal is about. the-goal-cross-cross.md names "pxx hosts itself somewhere that is not Linux/x86-64" as one of the two proofs, and xtensa is Track S's primary target. A defect whose only victim is the self-hosted compiler on the goal target is not a small defect; it is a narrow one.

What keeps it at 35 rather than higher is that nothing is blocked: --xtensa-long-calls builds it today. This is about the default, and about what the flag costs — the long form at every forward call site, "bigger and slower", which matters more on a flash-constrained target than it does on a host.

The design note, sharpened by today's failure

The banked negative stands as written: "A veneer has to be reserved during emission to be reachable, which is the same information problem as the call it would fix." But the failure above sharpens what "reserved" has to mean, and it is cheaper than the ticket assumed.

The flag reserves the long form per call site. A veneer reserved per calling body, per distinct forward callee is strictly cheaper — one slot at the end of a body serves every forward call to that callee from within it — and CALL0's ±512 KiB reach means the end of the calling body is always within reach of a call site inside it, which is exactly what was NOT true for the jump case (J reaches ±128 KiB). That is still a reservation and so does not escape the banked negative; what it escapes is the per-site cost that makes the flag expensive.

Untried. Not started here, because it is bigger than the tail of a session and this ticket is not blocking anything. Recorded so the next attempt starts from the sharpened version rather than re-deriving it.

2026-09-06 — the row that used to assert the wall no longer does, and nothing replaced it

Measured at eb8f1bb1c (compiler/pascal26 = 2945de63e58b5f23, srchash 8e4a68cd94d88acf matching the tree, real converged after 1 round(s)).

f49c0e11f fixed the epilogue case, correctly said "NOT A COMPLETE FIX", and added test_xtensa_finalizer_call_reach for what it fixed. What it did not do is notice that the existing xt_bigcall row asserted the refusal it had just made unreachable. That row has failed the full tier ever since (test-xtensa#147, red in every report from 2026-09-05 19:48 onward).

xt_bigcall generates 5000 straight-line s := s + 1; statements. The only over-512 KiB forward call a program of that shape makes is the exit path's call to __pxx_run_finalizers — precisely the call f49c0e11f now reserves the long form for unconditionally. The refusal could never fire there again.

The generator was never the limit, which is worth recording because the row's own comment guessed the other way ("a program that no longer crosses the bound"): at 40000 statements it emits 3.9 MB of code, 7.4x the bound, and still builds clean and runs correctly with no flag, output identical to the native build.

The residual, and it is this ticket's. A forward call between two ordinary procs more than 512 KiB apart still refuses — XtensaCallReaches still carries the Error naming --xtensa-long-calls — and no row anywhere reaches it now. The sharper half: the old row never reached it either. It was hitting the epilogue call the whole time and being read as the general case, so the coverage this ticket believed it had was already the narrow case before f49c0e11f removed it. Constructing the general shape needs two ordinary bodies placed that far apart, which is emission-order knowledge the generator does not have.

The row is repaired into its inverse (build with no flag, run, compare against the native build), so it now guards f49c0e11f against regression — strictly more than it guarded before, but not the same thing.

Date the gap correctly, because the obvious reading is wrong. Nothing about f49c0e11f created this coverage hole and nothing about the repair closes it. The general two-ordinary-procs case has never had a row: xt_bigcall was reaching the epilogue call from the day it was written. The gap is as old as the row, not as old as yesterday's fix.

2026-09-19 (frankS) — a second program meets it, and it is an application

examples/esp32/nilpy-s3: the ESP32-S3 NilPy demo. Without the flag the build refuses a forward call past CALL8's reach (image ~2.9 MB of code); with it, the program runs under qemu with output == CPython. Summary rewritten to the mechanism so it stops depending on how many programs happen to meet it today. The prio stays where the owner held it; the premise of that hold ("a population of one") is what changed, and that is reported to the coordinator, not acted on.