The gate, and why it is still there

pasparser_decl.inc widens the type NAMES widestring / unicodestring only when the program defines PXX_WIDE_PAYLOAD. Measured 2026-08-30 on 'café':

no define:   Length(w)=5  w[4]=195     UTF-8 bytes, the historic alias
with define: Length(w)=4  w[4]=233     UTF-16 units, é = U+00E9

Both are self-consistent. Neither is a bug. The default is the behaviour every existing pxx program was written against.

The gate does NOT hold back the thing the campaign was for. WideChar is not gated, so WideChar(u1) + WideChar(u2) is a genuine two-unit UTF-16 value in a default build and test_widestring_jsonscanner_wall — the fcl-json escape path, the campaign's whole reason to exist — passes ungated, byte-identical to FPC. That was checked before the campaign closed and is why the gate could stay.

Why this is a ticket and not a sentence in a closed write-up

Filed at the coordinator's insistence and the reasoning is worth keeping: a deliberate residue named only inside a done/ file is invisible to the ranker, invisible to check, and read by nobody. In three weeks an agent finds PasDefineExists('PXX_WIDE_PAYLOAD') and cannot tell deliberate from forgotten — and the cheapest reading of an undocumented gate is always "someone did not finish". This ticket is the thing that answers them.

What would retire it — a measurement, not a judgment

The gate was correct when written because breaking the alias made the COMPILER believe UTF-16 while the RUNTIME still stored UTF-8. That is no longer true; the lowering exists. So the remaining question is only blast radius, and it is answerable:

1. Force the define on globally and build the corpus + lib/** + examples/**.
2. The exposure to look for is code that declares WideString and then treats it as bytes — Length as a byte count, s[i] as a byte, a SizeOf-shaped assumption, anything passing it to a byte-oriented API.
3. Do not use an ASCII test to decide this. On ASCII a UTF-8 byte count and a UTF-16 unit count are the same number, so an ASCII sweep answers green for both settings and proves nothing. Three bugs in this campaign survived on exactly that.
4. Oracle: FPC with uses cwstring. Its default widestring manager converts byte-for-byte regardless of {$codepage utf8} or -FcUTF8, so stock FPC will report divergences that are not divergences.

If (2) finds nothing, the define comes off and WideString means UTF-16 everywhere. If it finds a handful, they are ordinary bug tickets and this stays open behind them. If it finds a lot, the gate was load-bearing and should be documented as permanent rather than removed.

Blocked by nothing. Low prio on purpose: nothing is broken today in either setting, and the campaign's goal is met in the default one.

Step 1 was BLOCKED by a bug, and the bug is now fixed (frank-rust, 2026-08-30)

This ticket's step 1 is "force the define on globally and build the corpus + lib/** + examples/**". It could not start. Under the define, this four-line program was a hard compiler error:

{$define PXX_WIDE_PAYLOAD}
program p; uses SysUtils; begin WriteLn(1); end.

compiler error: UTF-16 width conversion needs builtinwide (widestring with no RTL?)

lib/rtl/sysutils.pas has widestring functions, so under the define its own body needs a transcoder — and builtinwide is pulled by a token scan in pasparser_prog.inc that sees the PROGRAM's tokens and not its used units'. Adding var w: WideString to the program made the identical build succeed, which is what identified the scan's SCOPE as the cause rather than anything in sysutils. IRStrWidthConv's comment called this "unreachable through a normal compile", which was true before the define existed and false after it.

The define gated its own blocker, which is why this went unmet: without it widestring IS ansistring, dstWide = srcWide, and no conversion is ever synthesised. Nothing could find this except an attempt to do exactly step 1.

That is worth separating from an ordinary coverage gap when this decision is weighed (frankA's framing, and it is the better one): this is not a missing test, it is a region of the product no test could reach from outside the define. Every suite we run is a default build, so the entire PXX_WIDE_PAYLOAD side is unexercised by construction — and the one bug found there so far was found by trying to use it, not by testing it. Whatever else step 1 reports, it should be read as first contact with that side, not as a regression sweep over it.

Fixed: under PXX_WIDE_PAYLOAD the driver pulls builtinwide unconditionally. That is the same trade the scan's own note already makes — "a false positive costs 4 KB" — at a setting where it is not close, since the alternative is scanning every used unit's source before parsing it. Default builds are byte-identical: measured, uses SysUtils without the define produces the same 286488B of code as before. Test test/test_wide_payload_pulls_builtinwide_for_a_used_unit.pas, whose body deliberately does not name WideString, because naming it is what used to be required.

What step 1 now actually reports, on one corpus

fcl-xml's DOM (uses dom;), which is rung 3 of [[feature-pascal-corpus-oop]]:

So on this corpus the define is not costing anything and is buying 475 lines: it goes further with the define than without it. That is one data point, not the sweep step 1 asks for — and note it is an ASCII-clean measurement of reach, not of correctness, so it does not touch this ticket's rule 3.

A new consumer, which raises the value of retiring the gate

[[bug-p-a-property-in-an-interface-declaration-is-rejected]]'s corpus work added PWideChar(x) as a cast, and it is allowed only under the define — measured, not assumed. On 'hi' in a default build w[1] is 104 in both pxx and FPC (indexing steps one byte through the UTF-8 payload and widens, right for ASCII by construction), but PWideChar(w)[0] steps TWO bytes and yields 26984 ($6968, the pair packed) where FPC gives 104. The cast does not inherit the existing divergence — it introduces a NEW one, on plain ASCII. So it refuses without the define rather than producing a plausible wrong value.

That makes PWideChar the first construct that is flatly unavailable in a default build rather than merely different in it, which is a change in kind for this decision: previously both settings were self-consistent and neither was a bug.

A THIRD consumer, and this one is a conformance row that PASSES under the define (frankA, 2026-09-05)

The section above records PWideChar as "the first construct that is flatly unavailable in a default build rather than merely different in it". There are now two more, and they are a different kind of evidence again — not a construct we refuse, but valid FPC programs pxx cannot express at all in a default build:

tover1 declares test_string for shortstring, widestring, ansistring and pchar. Without the define widestring IS ansistring, so two of those four are the same type declared twice — pxx warns "duplicate definition", writes the second body into the first's row, and the program self-reports Failure. There is no way to write that program correctly here while the alias stands, and FPC's own test suite is where it comes from.

That is materially stronger than the earlier data points: the previous ones said the two settings differ, and this one says the default setting cannot represent a distinction the language has. It is not an argument that the gate is wrong — the blast-radius sweep this ticket asks for is still the thing that decides — but it belongs on the ledger, and rule 3 does not apply to it: this is not an ASCII-vs-non-ASCII question at all, it is a TYPE-IDENTITY question, and it would read the same on pure ASCII.

What is NOT part of this decision, and was fixed rather than deferred: the overload KEY itself was blind to the width — FindProcOverloadRec compared TTypeKind and tyAnsiString covers all three managed spellings — so even under the define the two overloads collided. That is a defect in either setting and it is now fixed (MatchParamExact and the registration check both read ProcParamStrElemTk; test_overload_widestring_and_ansistring_are_two_overloads carries the define for exactly the reason this ticket exists). So when the sweep runs, the overload machinery is no longer a confounder.

STEP 1 HAS AN ANSWER (frankS, 2026-09-07, compiler 19bee89a03e6)

-dPXX_WIDE_PAYLOAD works on the command line, and the conformance runner takes the compiler as its first positional — so a two-line wrapper forces the define across the whole corpus without editing the runner (which would be touching the instrument mid-measurement). Wrapper's own positive control: a {$ifdef PXX_WIDE_PAYLOAD} probe prints 0 bare and 1 through the wrapper.

Result: 406 pass, 5 fail, 89 skip, 50 auto-gated (of 550), against a 411 / 0 / 89 baseline at the same binary.

FAIL tcase0.pp  — exit 12 (want 0)
FAIL tcase12.pp — exit 1  (want 0)
FAIL tcase13.pp — exit 1  (want 0)
FAIL tcase28.pp — exit 1  (want 0)
FAIL tcase29.pp — exit 1  (want 0)

All five are RUNTIME failures, not compile errors, and all five are the same family. tcase0.pp declares ws: widestring; us: unicodestring and cases on them: exit 0 today, exit 12 — its own failure halt — under the define. That is step 2's target exactly: code that declares WideString and then treats it as bytes.

What this decides, and what it does not

By this ticket's own rule — "if it finds a handful, they are ordinary bug tickets and this stays open behind them" — the gate stays, and the tcase* family is the blocker to file behind it.

It does NOT clear the ticket, and step 3's warning is why. The corpus is overwhelmingly ASCII, and on ASCII a UTF-8 byte count and a UTF-16 unit count are the same number — so the 406 GREEN rows are not evidence that the define is safe for them. A red here is informative (those five really do change behaviour); a green here is not. Steps 2 and 4 over lib/** and examples/** with non-ASCII data are still unrun.

The oracle in step 4 needs re-examining before anyone leans on it

Measured: uses cwstring with w := 'caf' + #$C3 + #$A9 gives fpc Length=5, w[4]=195 — i.e. byte-for-byte, the same answer as pxx's DEFAULT and not the define's. That is the behaviour this ticket already warns about ("converts byte-for-byte regardless of {$codepage utf8} or -FcUTF8, so stock FPC will report divergences that are not divergences"), so it is consistent — but it means the cwstring recipe as written did not give a converting manager here, and whoever runs step 4 needs to establish one that does before treating fpc as an oracle. pxx cannot run the comparison program at all: uses cwstring is unit source not found, so the differential needs a different shape than "same source, two compilers".

What blocks the rows

tover1.pp passes under the define and cannot pass without it — without it widestring IS ansistring, so its two overloads are genuinely one signature (pxx says so: duplicate definition of 'test_string' with the same parameter types) and selection cannot distinguish them. tstring11.pp needs the define AND an open-array conversion. tstring10.pp needs the define AND punicodechar, which is undefined even under it. So this gate is the shared root of three gap: rows, and none of the three can burn while it stands.