← board

RemObjects Pascal Script — compile under pxx (embeddable scripting)

Why this is a good test case (the actual motivation)

Two wins at once:

  1. Compiler conformance. A self-contained, FPC-clean, mid-size Object Pascal codebase (lexer + compiler + bytecode runtime + import glue). Compiling it on the pinned stable is a heavyweight real-world test that exercises the dialect far past our own RTL — like Synapse, but a different shape (interpreter, not networking). Lowest-friction of the Pascal scripting engines, so it goes first.
  2. A feature for free. Once it builds, frank2 apps gain an embedded Object-Pascal scripting engine. We are not rolling our own — purely reusing.

Approach

Done when

$(PXX_STABLE) builds the Pascal Script core, and a frank2 host program runs a small script end-to-end (compile → execute → observe output) under a smoke. Stretch: host↔script binding of a hand-registered function.

License compliance (we honour it)

If we ship a demo or test app built on Pascal Script, we follow the license and give the attribution — a visible "made using RemObjects Pascal Script" line (and where to find it) in the app's aboutbox / docs / README, and we keep the upstream notice in any vendored source. Fair trade for a free engine; bake the credit line into the demo from the start, not as an afterthought.

Log

First probe 2026-06-28 (v83, --mimic-fpc)

Clone at external/pascalscript/ (remobjects/pascalscript, shallow). Core units probed with temporary lowercase copies (see [[bug-c-header-case-sensitivity-lookup]] — compiler lowercases unit name for file lookup; uPS* units have mixed-case filenames → not found without workaround).

unit state
uPSUtils [[bug-consteval-named-type-cast]] — IPointer(expr) in const expr fails ConstEval (same bug as Synapse TSocket(NOT(0)))
uPSPreProcessor same — IPointer cast
uPSCompiler same — IPointer cast
uPSRuntime [[bug-mimic-fpc-version-defines-missing]] — {$IF DEFINED(FPC) and (FPC_VERSION >= 3)} fails; FPC_VERSION not defined as integer under --mimic-fpc

3 Track A bugs gate the core (1 shared with Synapse, 1 new, 1 infrastructure):

  1. [[bug-c-header-case-sensitivity-lookup]] — unit name lowercasing blocks all uPS* units on Linux
  2. [[bug-consteval-named-type-cast]] — IPointer(expr) in const, blocks uPSUtils/uPSPreProcessor/uPSCompiler
  3. [[bug-mimic-fpc-version-defines-missing]] — FPC_VERSION integer missing, blocks uPSRuntime

When Track A fixes these, re-probe for the next wall.

Open questions

Probe log 2026-07-12 (opus-p)

Clone at github.com/remobjects/pascalscript, probe --mimic-fpc -Fu<clone>/Source -Fulib/rtl -Fulib/rtl/platform/posix, target unit uPSUtils. Walls burned this session:

  1. const array-of-RECORD with named-field element inits ((name: 'AND'; c: CSTII_and) keyword table) — LANDED (parser, test_const_array_of_record).
  2. SysUtils.CurrToStr / Currency — LANDED (sysutils shim: Currency=Double).
  3. Pos(tbtstring(' '), s) — string-typed ALIAS casts were pointer reinterprets (arg matched nothing) — LANDED: value no-op passthrough.

Current wall: CheckReserved(FLastUpToken, CurrTokenId) — a managed (tyAnsiString) field passed to a Const S: ShortString param: the const frozen-string param is by-ref for ABI, the managed→frozen conversion produces a non-lvalue, and the by-ref argument check rejects it. Needs the const-frozen-string param path to materialize a conversion temp (mirror the const-record temp rule) — parser/ir slice, file/pick up next session.

Probe log 2026-07-12 (later, opus-p)

uPSUtils compiles (walls 4-8): FPC variable typecast as var arg (Cardinal(len)), Dec(Byte(p^),32) cast-deref/type-keyword targets, TObject(x).Free statement, FreeAndNil, managed→ShortString param conversion.

uPSCompiler wall: IUnknown_Guid: TGuid = (D1:0; ...; D4:($C0,...)) — pxx has NO builtin TGuid record (it's a System type; interfaces reference it). The array-valued-field record const shape itself now works (LANDED, test_record_const_array_field — TGuid's D4 array field). What's missing is the builtin TGuid type + interface-GUID semantics. -dPS_NOINTERFACES skips the GUID consts and reaches the next wall (uPSCompiler:1963, a Decl.Params shape). Pascal Script core (uPSCompiler+uPSRuntime) is a multi-wall haul past this — needs builtin TGuid, ole2/Variant surface (uPSRuntime uses ole2), and more. Parked; uPSUtils is the concrete milestone reached.

Update: builtin TGuid landed (RegisterBuiltinTGuid — System record, SizeOf 16); uPSCompiler advances past the GUID consts to uPSCompiler:1963, a {$IFDEF CPU64}...Result := False block (CPU64 IS defined; the wall is the surrounding record-field expression Decl.Params[i].Mode shape after include expansion — needs isolation). Full Pascal Script core remains a multi-wall haul (ole2/Variant for uPSRuntime, InvokeCall.inc assembly). The generally-useful spinoffs all landed: array-field record consts, builtin TGuid, variable typecasts as var args, cast-deref Dec targets, TObject(x).Free, managed→ShortString param conversion, FreeAndNil.

2026-09-01 (frankH) — attempted; the wall is three bricks and two of them are one cause

Probed against a --depth 1 clone outside the repo with -Fups/Source, which is the reversible half of this ticket's own two options. Nothing was vendored, so there is no licence obligation incurred and nothing to revert.

The good news first, because it changes what this ticket is worth: uPSUtils compiles CLEAN on $(PXX_STABLE), first try, with no flags beyond -Mobjfpc and no source edits. That is a 40KB unit of real third-party Object Pascal, and it says the dialect surface is closer than "vendor it and see" suggests.

The three walls in uPSCompiler.pas, in the order the target hits them

1. unknown type: PByteArray (line 3085) — FIXED. FPC and Delphi declare TByteArray / PByteArray in SYSTEM, so real code reaches them with no uses clause. pxx has no System unit, so — exactly as the HModule note in that file already records — implicitly-reached System types live in lib/rtl/sysutils.pas. Added there.

The interesting part is a name collision that is deliberate and was checked rather than reasoned about: lib/rtl/hashing.pas already declares TByteArray = array of Byte, a DYNAMIC array — a different type wearing the same name. FPC has the same situation and resolves it the same way (a unit's own declaration shadows System's). Verified by compiling a program that uses BOTH in one file and by re-running every existing TByteArray consumer in the tree (lib_base64, lib_png, lib_zlib, raytracer) — lib_png's output is byte-identical to the same test built against the unmodified sysutils, which is the control that matters, since its last line reads bad chunk crc and would otherwise look like a regression I had caused.

2. expected comma or close parenthesis (line 1930) — FILED. tbtwidestring(p^.twidestring)[1] as a call argument. 13 occurrences of the shape in that one file.

3. SetLength expects a string variable in IR codegen (line 2753) — SAME BUG. SetLength(tbtstring(vari^.tstring), n). Confirmed same cause with a control: drop the cast and it works — SetLength(p^.s, 4) compiles, SetLength(tbtstring(p^.s), 8) does not.

Both are [[bug-p-a-cast-to-a-string-alias-silently-drops-a-following-index]]: a value cast to a non-pointer type is not transparent to the postfix tail. Its worst face is not either of these — in plain assignment position the index is silently DROPPED and FPC disagrees with no diagnostic, which is why that ticket is 60 and its two already-closed siblings were 45. Root cause is [[refactor-p-one-lvalue-path-for-statements-and-expressions]], which now has four instances.

Not reached

uPSRuntime stops earlier and for an unrelated reason — conditional directive: comparison requires integer operands — which has not been characterised. Probing was bounded deliberately: past wall 3 I would be editing the vendored source to keep going, and a wall map built on a source nobody else has is worth less than three walls anybody can reproduce.

What this ticket needs next

The dialect gap is smaller than the ticket assumed and it is concentrated: land the cast-transparency refactor and re-probe. That is one change, it clears two of three walls here, and it closes four tickets elsewhere. Vendoring and the smoke test are downstream of that, not of a long tail of small gaps.

2026-09-02 (frankH) — wall 2 is down; wall 3 is a different arm and is the only one left

[[bug-p-a-cast-to-a-string-alias-silently-drops-a-following-index]] is fixed (9339d6661). Verified on the exact shape this file uses rather than on the ticket's abstraction of it:

type tbtwidestring = WideString;
     PRec = ^TRec; TRec = record twidestring: tbtwidestring; end;
c := tbtwidestring(p^.twidestring)[1];      { was: expected ')' before '[' }
TakeW(tbtwidestring(p^.twidestring)[2]);    { was: expected comma or close paren }

Both compile and run and answer what fpc 3.2.2 answers. The pinned compiler refuses both. That is uPSCompiler.pas:1930 and the 13 occurrences of the shape in that one file.

Wall 3 is NOT the same bug and must not be assumed to have gone with it. SetLength(tbtstring(p^.tstring), 8) still answers "SetLength expects a string variable in IR codegen" — measured today, after the fix. It is a different arm: the specialId = 101 lowering requires an IR_LEA target and a cast node is not one, so this is the SetLength lvalue path rather than the postfix tail. The minimal repro is the three lines above with the string field. Whoever takes it should file or fix it in Track A/P and link it here; that is the residual question this note is naming an owner for.

Still not vendored, still no clone in the tree — the numbers above come from minimal repros of the shapes the 2026-09-01 attempt recorded, not from a fresh build of the upstream file. Re-running the real attempt is the next step and it needs the clone back.

2026-09-02 (frankH, later) — wall 3 is down; all three named walls are closed

SetLength(tbtstring(p^.tstring), n) compiles and runs (1fd4e7f22). It was a different arm from wall 2 and measurement made it bigger than the ticket said: all three spellings failed, each differently, and fpc 3.2.2 accepts all three.

SetLength(AnsiString(s), 7)     undefined variable (AnsiString)   -- at PARSE time
SetLength(tbtstring(s), 2)      SetLength expects a string variable in IR codegen
SetLength(tbtstring(p^.s), 8)   ...the same, through a record field

The builtin spelling dying in the parser and the alias ones dying in IR codegen is why it read as two problems: the target name goes to FindVarSym, misses, and the two populations diverge from there. Recorded because the ticket named only the alias-through-a-field form, and a reader fixing exactly that would have left the builtin spelling broken.

What this does NOT establish

Nobody has rebuilt uPSCompiler.pas since. There is no clone in the tree (and the 2026-09-01 attempt deliberately did not vendor one), so the three walls are closed as shapes, verified against minimal repros of what that attempt recorded — not against the file. A fourth wall behind the third is entirely possible and would be the ordinary outcome; the previous attempt found three by walking, not by predicting.

So the residual question has an owner: re-clone and re-run. That is the next step on this ticket and it needs network access the fixing sessions did not use. Until someone does it, the honest claim is "the three known walls are gone", not "Pascal Script compiles".

2026-09-02 (frankH, corrected by re-running the REAL file) — wall 3 is still up, and it is a different animal

The claim in the entry above that "all three walls are down" was wrong, and the thing that caught it was re-running the attempt against uPSCompiler.pas itself instead of against my repro of it. Recorded rather than quietly edited, because the way it was wrong is the reusable part.

1fd4e7f22 is a real fix — SetLength(AnsiString(s), n) and SetLength(TS(p^.s), n) are three spellings fpc 3.2.2 accepts and pxx rejected, and they now work. They are just not this file's shape. The repro was built from the ticket's own description of line 2753, and the description omitted the one fact that decides it:

  TIfRVariant = record ... case Byte of
    10: (tstring: Pointer);        { uPSCompiler.pas:147 }
  ...
  SetLength(tbtstring(vari^.tstring), TPSSetType(FType).ByteSize);

tstring is a Pointer, not a string. tbtstring(vari^.tstring) is therefore a genuine REINTERPRET — take the 8-byte slot and use it as a managed-string handle — and not the value-level no-op that a cast over a string lvalue is. My fix correctly declines it (its guard reads the operand's kind), so the attempt still stops at 2753, now for the honest reason rather than the parse one.

The size of the real wall, measured

tbtstring( appears 93 times in uPSCompiler.pas, 67 of them wrapping a plain lvalue. This is not a corner: it is how this codebase stores every string, and uPSRuntime.pas does the same (tbtstring(dest^) := ..., tbtstring(temp.Dta^)[i] := ..., tbtstring(cp^) := ...). So the capability needed is one thing stated once:

a POINTER-typed lvalue cast to a managed-string type is a string VARIABLE — readable, indexable, assignable, and resizable — with the refcount protocol applying to that slot.

FPC supports it and this is the standard Delphi idiom for a string inside a variant record. It is a Track A/P feature, not a parser patch, and it is the whole remaining distance to phase 1.

The lesson, which is this repo's own rule

A repro built from a ticket's PROSE is a repro of the prose. The ticket said "SetLength(tbtstring(vari^.tstring), n)" and I reproduced exactly that text with tstring declared as a string — the one substitution that made it a different bug. The file was three commands away the whole time. Re-run the target, not the description of it.

Re-checked against the file 2026-09-02 (second session)

The correction above was checked rather than taken on trust, against a fresh --depth 1 clone. Every load-bearing claim holds.

What makes the rows below worth anything is that each one is a MEASUREMENT against the real file — not that a second session agreed. This heading said "independently re-derived ... different reasoning" until frankA pointed out that this credits the wrong thing: a re-derivation that agrees is weak evidence when both derivations come from the same kind of reasoner working from the same source, and a restarted session inheriting an argument is exactly the case where agreement feels like corroboration and is not. The claim scopes to what was checked, which is the table:

claim check
10: (tstring: Pointer) in the variant record uPSCompiler.pas:147, verbatim
line 2753 is the SetLength on tbtstring(vari^.tstring) verbatim
93 uses in uPSCompiler.pas 93 lines match tbtstring( (96 occurrences; 78 wrap a plain lvalue)
uPSRuntime.pas does the same 45 uses, incl. tbtstring(dest^) := ... (4762) and tbtstring(temp.Dta^)[i] (9553)
the attempt still stops at 2753 reproduced, message verbatim

The repro, which the ticket did not carry — this is the reusable part:

git clone --depth 1 https://github.com/remobjects/pascalscript.git ps   # outside the repo
printf 'program drv;\nuses uPSCompiler;\nbegin\nend.\n' > drv.pas       # pxx has no standalone-unit output
./compiler/pascal26 --mimic-fpc -Mobjfpc \
    -Fups/Source -Fulib/rtl -Fulib/rtl/platform/posix drv.pas drv

pascal26:2753: error: SetLength expects a string variable in IR codegen (compiler a6207eb98ae6, tree at 3b04e6e19).

A second, free finding from the same run: the PIN cannot reach 2753 at all. stable_linux_amd64/default/pinned stops 823 lines earlier, at uPSCompiler.pas:1930, on tbtwidestring(p^.twidestring)[1] — wall (2). So the two fixes that landed on 2026-09-02 are load-bearing for getting this far and are not yet in any $(PXX_STABLE) consumer's hands. Anyone re-running this must build compiler/pascal26; probing the pin reproduces the OLD wall and would read as a regression.

Note that 1930 is the same animal as 2753 — twidestring is 18: (twidestring: Pointer), one line below tstring in the same variant record. Wall (2) and wall (3) were always ONE capability seen through two statements; the index arm happened to be reachable without it and the SetLength arm is not.

RE-MEASURED 2026-09-05 — premise CURRENT, and the ticket records only the LOUD half

Reproduced exactly as written: the attempt stops at SetLength expects a string variable in IR codegen. Wall (3) is real and unchanged. What follows is the part the ticket did not have.

The boundary, by varying the operand rather than the access path. Nine probes, type t = AnsiString:

operand of t(...) in SetLength(t(...), 2) result
s: AnsiString (no cast at all) compiles, ab
s: AnsiString (cast) compiles, ab
p^ where p: ^AnsiString compiles, ab
record field of type AnsiString compiles, ab
p: Pointer refused
Pointer field, direct refused
Pointer field via a record pointer refused

So the discriminator is the kind of the slot, not the indirection and not the record — every string-typed slot works. That matters for this ticket because uPSCompiler.pas stores strings in Pointer variant arms exclusively; there is no partial-success path where some of the 93 sites work.

THE SILENT HALF, which is not in the summary above and is worse than the refusal. The same cast in an rvalue position does not refuse — it returns the POINTER:

t(p) := 'abc'; writeln(t(p));   pxx: 4261104     fpc: abc
Length(t(p))                    pxx: 4265208     fpc: 3
p = Pointer(s)                  pxx: TRUE        fpc: TRUE
Length(s)                       pxx: 3           fpc: 3

The STORE is correct — the pointer in the slot is the right pointer. Every READ through the cast is wrong, with no diagnostic. A vendored uPSCompiler would not have stopped at 2753; it would have compiled further and produced numbers where strings belong. A ticket blocked on a refusal is safer than the state after that refusal is lifted, so the two must be fixed together.

Filed as its own Track P ticket, per this repo's rule that a Track B ticket meeting a compiler gap files it in the owning lane: [[bug-p-a-string-alias-cast-over-a-pointer-slot-is-a-no-op-and-reads-the-pointer]]. That ticket carries the cause (the strAliasCast arm returns the operand with its own kind, correct for a string operand and a no-op for a pointer one), and the measured reason a parser-only fix is not enough: retagging the node made writeln print abc followed by out-of-bounds memory and left Length still answering the pointer, because the IR lowers the load from the symbol. Tried, measured, reverted — a wrong number is not improved by becoming an OOB read.

This ticket is now blocked-by that one rather than by an unlocated wall.

2026-09-06 (frankH) — three walls down, both survivors are compiler-lane

Picked up off ticket_age as the oldest unblocked row. The staleness check at the head found the blocked-by blocker in done/, which made the summary's headline claim false, so re-measuring was the deliverable.

Verified by running, not by reading the folder — done/ is a claim about the past and the repro is a claim about this binary:

rvalue : abc        (recorded: 4261104, the pointer as a number)
length : 3          (recorded: the pointer)
setlen : ab         (recorded: SetLength expects a string variable in IR codegen)
store  : TRUE

All four identical to fpc 3.2.2 -Mdelphi.

The wall ladder, uPSCompiler.pas

at wall
2753 string-alias cast over a pointer slot — fixed (9339d6661)
3776 on: unknown exception class (EZeroDivide) — fixed here (d4fe6ede3)
5031 @Func.Attributes.Items[i].AType.OnApplyAttributeToProc — open

The middle one was not a missing name. pxx had no float exception family: EInvalidOp descended from Exception, EZeroDivide/EOverflow/EUnderflow did not exist, and reZeroDivide/reOverflow/reUnderflow all raised integer classes — so on E: EMathError, which is how real code catches any float error and exactly what uPSCompiler writes, caught nothing at all. Silent, not a diagnostic. Fixed against the oracle, guarded by test/lib_math_exception_tree.pas, whose two FALSE rows are the actual test.

uPSRuntime — the {$IF} wall never existed under --mimic-fpc

This ticket recorded uPSRuntime as "not reached: it stops earlier on a {$IF} comparison". That was measured without the flag. With --mimic-fpc it parses to line 3049, exactly as [[feature-embed-dwscript-rtti]] predicted when it found the same {$IF CompilerVersion>21.0} refusal sitting inside {$IFNDEF FPC}. The real wall is a nested function read(...): Boolean — see [[bug-p-read-write-exit-and-halt-cannot-be-declared-as-user-routines]], where the five-name divergence set is enumerated and writeln is the parity row that stops the fix being "un-reserve the builtins".

What is left

Both remaining walls are Track P compiler gaps with reduced repros, and neither is a library gap. Nothing here needs lib/ work to proceed.

2026-09-06 (frankH), later — the @ wall closed and bought TWO LINES

[[bug-p-at-over-a-class-base-consumes-only-one-selector]] landed hours after it was filed. Verified by running the eight shapes against this binary (3bc524a975bd) rather than by reading done/ — all eight now compile, including the four that failed.

uPSCompiler's wall moved 5031 -> 5033. Two lines, and the new one is the same construct read the other way:

if @Func.Attributes.Items[i].AType.OnApplyAttributeToProc <> nil then   { 5031, fixed }
  if not Func.Attributes.Items[i].AType.OnApplyAttributeToProc(          { 5033, open }
       Self, Func, Func.Attributes.Items[i]) then

Taking the address and calling are one construct; only the address arm moved. Filed as [[bug-p-a-call-through-an-indexed-property-in-the-chain-does-not-resolve]] and now this ticket's blocked-by. The failing ingredient is the indexed property, not the chain — o.R.Ev(1) through a record field compiles.

Worth carrying to whoever fixes it: the statement-position diagnostic is o is not a procedure or function, naming the BASE for a callee four selectors away, so the message sends a reader to the wrong end of the expression.