← board

A foreign thread needs its own TLS block, and the bounds are the hard part

What is NOT the question

Detection. feature-a-io-lock-owner-from-tls-not-gettid shipped the one test inheritance cannot fake — a thread's own rsp against the bounds its block's owner recorded — and it is live at ir_codegen.inc:1099-1118. Anywhere that needs "am I running on somebody else's block" can ask, with two loads and two compares and no syscall.

[[bug-a-a-foreign-thread-shares-the-main-thread-s-heap-magazine]] lists "detect and install lazily" and dismisses it on the detection half. That analysis predates the discriminator and should not be read as current.

The question

Where does a lazily-installed block come from, and what bounds go in it?

Installing is easy: carve TLS_BLOCK_SIZE, zero it, arch_prctl(ARCH_SET_GS). Everything downstream then works — a private exception chain, a private heap magazine, private signal slots — because every consumer reads gs: and asks no further questions.

The bounds are what bite, and they bite in a loop:

A fourth shape worth weighing: a separate idempotence marker that is not the bounds — e.g. the block's own address compared against a value only the installer could have written there, or a tid cached once at install and checked only on the install path (where one gettid per thread is free) rather than on every gs: read.

Where to hook it

Also undecided, and cheaper: try entry alone bootstraps the whole block for any thread that ever raises, but not for one that only allocates. The honest set is probably try entry plus the heap fast path.

Why it matters now

[[the-goal-cross-cross]] names DOSBox. DOSBox, SDL, GTK and every threaded C library create threads pxx never sees, and [[bug-a-the-exception-chain-fix-is-defeated-by-a-libc-pthread]] is a measured crash today: 3 runs of 3 print Unhandled exception where the identical work on one thread in the same binary is clean.

2026-09-02 (frankA) — measured: glibc packs thread stacks with NO margin, which refutes two of the four options by arithmetic

The bounds question has a number attached to it and nobody had taken it. On this box (glibc, ulimit -s 8192):

main    rsp=0x7ffce34e32cb
worker 0  hi=0x79d1bd1fee83  lo=0x79d1bcad9c50  own-range=7492147 bytes
worker 1  hi=0x79d1bc9fde83  lo=0x79d1bc2d8c50  own-range=7492147 bytes
worker 2  hi=0x79d1bc1fce83  lo=0x79d1bbad7c50  own-range=7492147 bytes
gap w0->w1 = 8392704 bytes
gap w1->w2 = 8392704 bytes
MARGIN between w1's own observed range and w2's top = 900557 bytes

Consecutive live pthread stacks are exactly 8392704 bytes apart — 8MB plus one 8KB guard page — and they are CONTIGUOUS, not scattered. main is about 140GB away from all three, which is why the shipped I/O-lock discriminator works today: it only ever has to tell main from a worker, and pxx-created threads get exact bounds from the clone stub.

This refutes the guessed bounds, by their own numbers

The option above proposes rsp - 8MB .. rsp + 64KB. 8MB is exactly the stack size, so from a shallow rsp that window's lower edge lands in the NEIGHBOUR's stack — the failure the parent ticket calls "the harder failure", reached not by an unusual thread but by the default one. Any guess large enough to cover a full stack is large enough to cover the next thread's.

And it refutes the widening-window shape, which is worth stating because it looks sound

A fourth-option variant that suggests itself: install with the bounds unset, then instead of reinstalling, WIDEN a [minObservedRsp, maxObservedRsp] window each time the thread is seen. It looks sound — different threads have disjoint stack VMAs, so a window covering only addresses this thread actually touched can never contain another thread's rsp.

The margin measured is 900557 bytes and that is an artefact of my probe stopping at 1800 frames, i.e. 7492147 of the 8388608 bytes available. A thread that uses its whole stack has a window that reaches the guard page, and the neighbour's stack begins immediately on the other side of it. The scheme is sound only while every thread stays strictly inside its stack, which is not a property the runtime can assume of a program it did not write. It also breaks outright on the sigaltstack, which the existing discriminator's comment already handles as a deliberate miss.

So the residual shrinks: an idempotence marker must be something other than an address range. The tid cached at install and compared only on the install path survives this measurement; the two address-based shapes do not.

The probe, and the confound it caught first

static void rec(int id, int d) {
  volatile char pad[4096];
  unsigned long r = (unsigned long)&pad[0];
  pad[0] = (char)d;
  if (r < lo_seen[id]) lo_seen[id] = r;
  if (r > hi_seen[id]) hi_seen[id] = r;
  if (d > 0) rec(id, d - 1);
}

with each worker recording hi/lo around rec(id, 1800).

Create all threads before joining any. The first version created and joined one at a time, and all three reported byte-identical hi and lo: glibc caches and reuses a dead thread's stack, so the gap this probe exists to measure was exactly zero and read as a finding. The reused stack is a real fact about glibc and it is not the fact the question needed.

Not a recommendation: no option is chosen here and no code changed. What is added is that two of the four are now arithmetically excluded rather than weighed.

2026-09-02 (frankA) — the rsp marker is defeated by pxx's OWN generators, and the widening-window exclusion was right for the wrong reason

Two measurements. The second matters more than the first.

1. The widening-window exclusion does not hold as stated

The section above excludes the widening [minObservedRsp, maxObservedRsp] marker because "a thread that uses its whole stack has a window that reaches the guard page, and the neighbour's stack begins immediately on the other side of it." Reaching the guard page and CONTAINING a neighbour's rsp are two different claims, and only the second is the failure. Measured, four pthreads each recursing until 64KB from their own known floor (pthread_getattr_np, so the stop is not a guess), all created before any is joined:

w0 vma=[0x7cd89fa00000,0x7cd8a0200000) size=8388608  seen span=8317056  unused-below-seen=64896
w1 vma=[0x7cd89f1ff000,0x7cd89f9ff000) size=8388608  seen span=8317056  unused-below-seen=64896
w2 vma=[0x7cd89e9fe000,0x7cd89f1fe000) size=8388608  seen span=8317056  unused-below-seen=64896
w3 vma=[0x7cd89e1fd000,0x7cd89e9fd000) size=8388608  seen span=8317056  unused-below-seen=64896
pairwise overlaps=0 of 6 pairs
positive control (w0 vs itself) overlap=1 (must be 1)

99.15% of each stack touched, zero overlaps. An observed window is a subset of its own VMA by construction, consecutive VMAs are separated by a 4096 guard page, so a neighbour's rsp cannot be inside one. The stack-REUSE hazard does not rescue the exclusion either: a thread's stack is allocated at pthread_create, i.e. while its creator is still alive, so a child cannot land on the parent's stack — tested directly, 3 runs of 3, the child got a different VMA every time (while four sequential create/join threads got the byte-identical VMA every time, which is the reuse the earlier probe tripped over).

This is not an argument to adopt the widening window. It is an argument that the reason recorded for excluding it was falsifiable and false, and a false exclusion in a decide ticket is worse than an open option: the next reader checks it, finds it wrong, and reopens the wrong door.

2. The reason that does hold, and it ships today

A pxx stackful generator body runs on a HEAP stack. lib/rtl/coroutine.pas CoAlloc does GetMem(CO_STACK) — 65536 bytes — builds the initial saved-state frame there, and CoSwitch runs the body on it. Measured:

procedure frame  = 140732946248072
generator frame  = 127672537645112
distance         = 13060408602960 bytes (~13 TB)

So inside a running generator, every rsp-against-stack-bounds test says "foreign", on the thread that owns the block, with no foreign thread anywhere in the program.

For the shipped I/O-lock discriminator this is harmless and already accounted for: a miss falls through to gettid, which is correct and slower. For an idempotence test the same miss is the unsafe direction — it would install a fresh TLS block on every generator entry and zero the live exception chain, which is exactly the failure this ticket attributes to HI = 0, reached not by an edge case but by a construct that ships, is documented (docs/library/async.md) and is tested (test_coswitch.pas, test_scheduler_exc.pas, test_residency_coswitch.pas).

And the two mechanisms already interact: CoAlloc writes exc_top := 0 into the generator's initial frame — "fresh chain on this stack". The exception chain is ALREADY per-stack rather than per-thread wherever a generator is involved, which is a second reason not to build its ownership test out of a stack address.

What this leaves

An idempotence marker must be something no stack switch can move. Every address-based candidate in this ticket is one that a stack switch moves, and pxx switches stacks itself. The gettid-at-install marker survives both measurements; rsp survives as a fast-path FILTER only, and only where a miss is fail-safe.

That also sharpens the hook question above. try entry was offered as the cheap hook — and generators are precisely where try/except runs on a stack the thread does not own, so the hook and the defeated marker meet at the same site.

Still not a recommendation: no option is chosen here and no code changed. Three of the address-based shapes are now excluded on a reason that is measured rather than argued, and one exclusion already in the ticket has been corrected.

2026-09-22 (frankh-c0) — WHAT A BLOCK COSTS, on the axis this ticket does NOT turn on

Bounded deliberately to the question asked — how many slots, how many bytes — and stopping there. It is derived from the definitions, not measured off a binary, and it is labelled as such.

part slots bytes
scalars (SELF, TID, STACK_LO/HI, 4 SIG, 4 EXC, MAGBUSY) 13 104
padding to the magazine (13..15) 3 24
heap magazine list heads (HEAP_MAG_BINS = 64) 64 512
heap magazine counts 64 512
slot map total (TLS_USER_FIRST_OFF, the source's own comment says 1152) 144 1,152
threadvar area (TLS_USER_BYTES_DEFAULT, assigned at ir_codegen.inc:1718) 3,072
TlsBlockSize = TLS_USER_FIRST_OFF + TlsUserBytes 4,224

TWO THINGS FALL OUT AND BOTH BEAR ON THE DECISION.

1. 89% of the slot map is the heap magazine — 1,024 of 1,152 bytes, 128 of 144 slots. That is exactly the resource bug-a-a-foreign-thread-shares-the-main-thread-s-heap-magazine is about. So "give a foreign thread its own block" and "stop a foreign thread sharing the magazine" are the same allocation, not two costs to weigh against each other.

2. 73% of the block is the threadvar area, and another open ticket says that area is usually unnecessary. feature-a-the-threadvar-area-is-3072-bytes-of-bss-in-every-program-that-has-no-threadvar (p70, track A, unowned) proposes sizing it on demand. If that lands, a foreign thread's block falls from 4,224 to ~1,152 bytes, and the knob already exists: PXX_TLS_USER_0 sets it to zero today (ir_codegen.inc:1720).

So the cost side of this fork is not fixed, and it is cheaper than it looks in exactly the programs that would hit it. A decision taken against 4,224 bytes is being taken against a number another ticket is already trying to move.

A FORK PRICED AGAINST A LIVE NUMBER ACQUIRES A DEPENDENCY ON NOBODY IMPROVING THAT NUMBER. Same family as a ticket summary citing a currently-firing row: the sentence is true when written and is falsified by someone else's fix, silently, in the direction that makes the decision look already-made. So whoever sends this up must send the RANGE — 4,224 today, ~1,152 if the threadvar area is sized on demand, and PXX_TLS_USER_0 proves the floor is reachable now — not the single number. Answering against 4,224 alone is answering a question that expires.

NOT MEASURED HERE, ON PURPOSE: the SETUP cost (what installing a block costs in instructions/time). The box was at load average 14+ and a timing row taken there would look like evidence — see the deferral note in perf-o-the-variant-hidden-dest-clear-....

RETIRED THE SAME DAY BY THE SECTION TWO BELOW, and this sentence said otherwise for hours after that section was written. It read: "that half is outstanding and is the only thing missing before the fork can be stated in goal terms with costs attached." That is false. The correction below establishes that this ticket does not turn on cost at all — its hard part is the idempotence MARKER — so a setup-cost number is not missing from the fork and never was blocking it. Setup cost becomes worth having only if the owner answers work, and then it is ordinary implementation cost, not a decision input.

Worth recording because of WHO was wrong about it: the same seat wrote the correction and then repeated the retired sentence three more times the same afternoon — to two peers and in its own status. Own text reads as already-checked, so the author is worse placed to notice than a stranger. CLAUDE.md's authorship is not protection arriving inside the one file where both halves were visible at once.

This is NOT an audit of the TLS machinery and must not become one.

Correction, same day, same seat: THE COSTING ABOVE PRICES AN AXIS THIS TICKET DOES NOT TURN ON

This section's heading said "so the fork is a yes/no and not an architecture question" for a few hours. That was written from what the costing was COMMISSIONED to do, before I re-read the fork, and it is wrong.

This ticket's own summary says the open question is where a lazily-installed block comes from AND what marker says "already mine", and the marker cannot be an address — because a pxx stackful generator runs on a heap stack 13TB from its thread's frame, so every rsp test reads a running generator as foreign. That is an idempotence problem, not a budget problem. Everything above makes the block cheaper and merges its cost with the magazine ticket's; none of it touches the marker. A reader who takes the table as answering the fork will conclude the fork is priced when its hard part has not been approached.

So the two findings stand as written and their SCOPE is narrower than the heading claimed: cost is no longer a fork — it is a range, 1,152 to 4,224 bytes, with a named mover — and that half is ours to close, not his.

What is left for the owner is one sentence with no implementation noun in it: do we want a thread that pxx never wrapped to WORK, or to be REFUSED loudly? If "work", the marker question is engineering and we own it. If "refuse loudly", the block question evaporates and so does this costing.

The general shape, which is why the correction is here rather than only in a message: a costing commissioned to settle a fork will be read as having settled it, whatever axis it actually priced. The heading is where that damage lands, because the heading is what a ranker and a skimming reader get. A section heading that states a CONCLUSION is the same animal as a probe labelled with its answer — it puts the claim upstream of the evidence, and re-reading has nothing to check it against. Name the measurement in a heading, never its consequence.

MEASURED, and it retires the RANGE the section above reports: the cost is 4,224 FLAT for any program this fork is about

The costing above says the threadvar area is a number "another ticket is moving", and gives the cost as a range of 1,152 to 4,224 bytes. That was written in the wrong tense and the range does not describe this fork's subject. Both halves corrected here, measured 2026-09-22 at HEAD by printing __pxxTlsBlockSize from compiled programs rather than by reading definitions.

program shape __pxxTlsBlockSize bss
no threadvar, no uses 1,152 35,324
a threadvar 4,224 38,400
no threadvar but WITH uses 4,224

The mover has already shipped. feature-a-the-threadvar-area-is-3072-bytes- of-bss-in-every-program-that-has-no-threadvar says so in the first four words of its own summary — ROUTES C AND A SHIPPED — and the section above cites that ticket while describing it as a proposal. Delta is exactly 3,072, chosen per program at compile time, today.

AND THE THIRD ROW MAKES THE CHEAP END UNREACHABLE HERE. The zero-area arm requires a program naming neither threadvar nor uses. A program that creates a thread cannot be one: BeginThread is undefined without uses (error: undefined variable (BeginThread)), as are TThreadID and WaitForThreadTerminate. Tested rather than asserted, because "a threaded program always has a uses clause" is exactly the quantifier that should not be written from intuition.

So the cost for this fork is 4,224 bytes per foreign thread, flat, today — not a range, and not a number anyone is currently moving. Quoting the range here would understate the cost by nearly 4x on precisely the programs at issue.

One thing survives and is worth keeping, because it makes 4,224 read as justified rather than arbitrary: the split is not a tuning knob, it tracks whether the program can have threadvars at all. 1,152 is the size when there is provably nothing to hold. The 3,072 is therefore not waste in the threaded case.

Scope of the measurement, stated so nobody reads it as wider: this is the Pascal arm. A foreign thread entering a unit-free pxx OBJECT from C would see 1,152; that is the object-consumer direction, it belongs to decide-a-is-a-pxx-object-a-self-contained-runtime-or-a-translation-unit, and it is not what "4,224 flat" is claiming.

The general shape, and it is why this is a correction and not an edit: the section above was accurate about definitions and wrong about the world, because it read a ticket's PROPOSAL framing past a summary whose first line said the work had shipped. A cost derived from constants cannot tell you which population reaches which constant — that takes a compiled program. Derive the bound, then compile something to find out who is standing under it.