TLS provider abstraction — pluggable backends (OpenSSL + handrolled)
- Type: feature (library / architecture) — the TLS seam
- Status: working
- Owner: frankH
- Opened: 2026-06-24
- Relation: the
https://enabler for [[feature-own-net-http-lib]]. Umbrella over two backends: [[feature-tls13-from-scratch]] (native) and the OpenSSL backend. That second one read "needs [[feature-real-dynlib-loader]]" until 2026-09-05; that ticket isdoneand its summary names the exact thing this one needed — "OpenSSL 3 loaded and answering" — so the dependency is satisfied, not merely closed. Checked rather than taken from the slug: the two items it leaves open are an arm32/aarch64 RUN and Synapse's own SSL path, neither of which this ticket goes through.
Slice 1 landed — the seam + plumbing proof (2026-06-25)
lib/rtl/tls.pas ships the backend-neutral contract: TTlsRole, TTlsResult
(tlsOk/tlsWantRead/tlsWantWrite/tlsClosed/tlsError), opaque TTlsConn,
and TTlsBackend (the vtable: Name / Handshake / Read / Write / Close).
Plus a process-global registry (TlsRegisterBackend / TlsActiveBackend /
TlsAvailable) and neutral wrappers (TlsHandshake / TlsRead / TlsWrite /
TlsClose) that fail cleanly with tlsError when no backend is registered
(never crash — the dynlibs-stub discipline). No backend ships here.
Signature refinement vs the sketch below: Handshake returns a TTlsResult with
the connection as a var c: TTlsConn out-param (uniform with Read/Write error
reporting), rather than returning TTlsConn directly.
Plumbing proven by test/lib_tls (14 checks, wired into make lib-test as
tls-seam): the no-backend path refuses cleanly, then a mock plaintext
backend (Read/Write just pass bytes over the fd) registered through the seam
carries a real loopback round-trip via TlsHandshake/TlsWrite/TlsRead/
TlsClose, and clearing the registry restores the clean state. Exercises the
vtable dispatch + registry independent of any crypto.
Slice 2 landed — http routes https:// through the seam (2026-06-25)
lib/rtl/http.pas now sends/receives every byte through a TLS-aware transport
funnel (HttpSendAll / HttpRecvSome / HttpTlsConnect / HttpIoWait): when a
URL is https:// it does TlsHandshake-after-connect and routes I/O via
TlsWrite/TlsRead, else the plain blocking (Net*) / reactor (Tcp*) path
exactly as before. Covers all four transports — blocking one-shot
(HttpRequest), async one-shot (HttpRequestAsync), keep-alive
(THttpConnection gained IsTls/Tls; HttpConnect/HttpConnectAsync take an
isTls arg; close tears the TLS layer down first), and the async pool (reuse
keyed on host:port:scheme so an https conn is never handed to a plain request).
The data-path Read/Write want-loop maps tlsWantRead/tlsWantWrite to
WaitReadable/WaitWritable (async) or PalPoll (blocking), so a backend that
would-block yields the coroutine and resumes — the async TLS path OpenSSL needs.
The handshake is taken as completing within one TlsHandshake call (the mock
- a blocking OpenSSL backend do; a fully-async handshake would need a resume step
added to the seam — noted as future work). With no backend, an
httpsrequest fails cleanly (Ok=False) — never crashes.
Proven by test/lib_https_mock (6 checks, wired into make lib-test as
https-mock-seam): no-backend https fails clean, then a mock plaintext backend +
loopback server let HttpGetAsync('https://...') complete through the seam over
the reactor (status 200 + body), exercising the want-read yield path. Real
crypto waits on the backends.
Slice 3 landed — OpenSSL backend, real HTTPS (2026-06-25)
lib/rtl/tls_openssl.pas implements TTlsBackend over libssl.so.3, dlopen'd at
runtime through the real loader ([[feature-real-dynlib-loader]], landed v68).
OpenSslTlsRegister loads the lib, resolves TLS_client_method / SSL_CTX_new /
SSL_new / SSL_set_fd / SSL_connect / SSL_read / SSL_write /
SSL_get_error / SSL_shutdown / SSL_ctrl (SNI via
SSL_CTRL_SET_TLSEXT_HOSTNAME) / frees, builds a client SSL_CTX, and registers
itself as the active backend. Entry points are plain (non-cdecl) proc vars — the
x86-64 default ABI matches SysV cdecl for these pointer/int signatures (same as
the dynlibs strlen smoke). SSL_get_error maps WANT_READ/WANT_WRITE →
tlsWantRead/tlsWantWrite, ZERO_RETURN → tlsClosed.
Verified end to end: make tls-openssl-devtest (tools/tls_openssl_devtest.sh
test/devtest_tls_openssl.pas) builds the client with-dPXX_DYNLIB_LIBC, spins up a loopbackopenssl s_server -wwwwith a self-signed cert, and a blockingHttpGet('https://…')returns status 200 + a ~5 KB decrypted body — a real TLS 1.3 handshake + GET through our http stack ⇄ OpenSSL. Opt-in / non-hermetic (needs the openssl CLI + libssl); skips cleanly when absent, so it is not in the core lib-test gate.
Slice 4 landed — async TLS handshake (2026-06-25)
The seam handshake is now non-blocking + resumable: Handshake does one step
(allocates the conn, attempts SSL_connect) and returns tlsOk / tlsWantRead /
tlsWantWrite / tlsError; a new TTlsBackend.HandshakeResume(c) (neutral
TlsHandshakeResume) does each subsequent step after the fd is ready. http's
HttpTlsConnect drives the loop — HttpIoWait between steps maps to PalPoll
(blocking) or WaitReadable/WaitWritable (async coroutine yield). A blocking fd
returns tlsOk on the first step (loop never runs), so the blocking path is
unchanged; a non-blocking reactor fd yields and resumes.
OpenSSL backend updated accordingly (no internal poll-loop in Handshake;
SslStepConnect shared by Handshake/HandshakeResume). Verified: the
tls-openssl-devtest now runs both a blocking HttpGet and an async
HttpGetAsync (coroutine, Spawn/RunUntilDone) over real openssl s_server —
both return 200 + decrypted body. So HTTPS now composes with the reactor.
Slice 5 landed — certificate verification + trust store (2026-06-25)
The OpenSSL backend is now secure by default. OpenSslTlsRegister loads the
system trust store (SSL_CTX_set_default_verify_paths), sets
SSL_VERIFY_PEER, and per connection calls SSL_set1_host(host) so the
handshake validates both the chain and the hostname (CN/SAN). A failed
verification aborts SSL_connect → the request returns Ok=False;
OpenSslTlsLastVerifyResult exposes the X509_V_* code.
OpenSslTlsRegisterEx(verifyPeer, caFile) adds a private/test CA on top of the
system store, or turns verification off (dev only).
Verified by the extended tls-openssl-devtest against openssl s_server
(self-signed, CN/SAN=localhost): reject — system-store-only, the untrusted
cert is refused (Ok=False, verify_result=18 = self-signed); accept — with
the test CA trusted, blocking GET → 200 and the hostname matches; async — the
verified connection also works via HttpGetAsync on the reactor.
Slice 6 landed — server side (SSL_accept) + OpenSSL⇄OpenSSL interop (2026-06-25)
The OpenSSL backend now plays both roles on one active backend. Handshake
branches on role: clients SSL_connect, servers SSL_accept (shared
SslStepHandshake, want-read/write handling identical so the async resume loop
serves both). OpenSslTlsServerInit(certFile, keyFile) builds a server SSL_CTX
(TLS_server_method + SSL_CTX_use_certificate_file/use_PrivateKey_file)
alongside the client ctx — a single process can serve and consume TLS at once.
Verified (devtest_tls_interop, run by tls-openssl-devtest): our
OpenSSL-backed HTTPS server (accept + seam handshake + TlsRead/TlsWrite)
⇄ our HttpGetAsync client, both coroutines on one reactor thread, client
verifying the server cert + hostname → status 200, body intact. So the
client×server interop holds for the OpenSSL backend on both ends (the diagonal +
the our-client⇄s_server cell from slice 3/5).
Remaining for the umbrella ticket: the native backend ([[feature-tls13-from-scratch]], deferred) — needed for the off-diagonal native⇄OpenSSL interop cells. The OpenSSL half of this ticket is functionally complete (client + server, blocking + async, verified).
Decision (2026-06-24)
Support both TLS backends from the start, behind one common interface:
- OpenSSL (dlopen
libssl/libcrypto) — the safe, well-tested, standard default. Production path. - Handrolled TLS 1.3 + kTLS ([[feature-tls13-from-scratch]]) — the platonic, syscall-only path; a superb real-world compiler stress test.
Why both, from the get-go (the user's reasoning):
- OpenSSL is the sane/safe default; rolling our own is the compiler test — keep both, don't pick.
- A common seam lets us mix and match in one app: e.g. an HTTP server on OpenSSL and an HTTP client on the native stack (and vice versa). That is the proof the abstraction is correct and an interop test — our handrolled stack must speak real TLS to OpenSSL on the other end, the strongest possible correctness check.
The seam
A backend-neutral TLS connection contract (e.g. lib/rtl/tls.pas) that any net
code (http, future servers) talks to instead of raw NetSend/NetRecv:
type
TTlsRole = (tlsClient, tlsServer);
TTlsResult = (tlsOk, tlsWantRead, tlsWantWrite, tlsClosed, tlsError);
{ A backend = a vtable/class implementing: }
TTlsBackend = class
function Handshake(fd: Integer; role: TTlsRole; const host: string): TTlsConn; virtual; abstract;
function Read (c: TTlsConn; buf: Pointer; len: Integer; var got: Integer): TTlsResult; virtual; abstract;
function Write(c: TTlsConn; buf: Pointer; len: Integer; var put: Integer): TTlsResult; virtual; abstract;
procedure Close(c: TTlsConn); virtual; abstract;
end;
- Async-aware:
Read/WritereturntlsWantRead/tlsWantWrite(OpenSSL'sWANT_READ/WANT_WRITE, the native stack's equivalent) so the async path maps them toWaitReadable/WaitWritableand yields the coroutine. The blocking path loops onPalPoll. Same contract serves both transports. - Selectable: a global default backend + per-connection override. Default = OpenSSL when its lib loads, else native (or configured).
http'sisTlsbranch routes its send/recv through the active backend; the pure build/parse helpers are unchanged.
Backends
tls_openssl.pas— thin binding overlibssl/libcryptovia the dynlib loader ([[feature-real-dynlib-loader]]).SSL_CTX_new/SSL_new/SSL_set_fd/SSL_connect/SSL_read/SSL_write, non-blocking +SSL_get_error→ WANT_READ/WRITE. This makes the dynlib loader a concrete, motivated consumer.tls_native.pas— wraps [[feature-tls13-from-scratch]] behind the same vtable. kTLS offload sits under it (per-platform), invisible to callers.
Testing (the payoff)
- Backend conformance: each backend does a real handshake to a known endpoint.
- Cross/interop matrix under
make lib-test(loopback, coroutine-driven):client \ server OpenSSL native OpenSSL sanity native-server correctness native native-client correctness pure-PXX e2e Each cell: handshake + a GET round-trip. The off-diagonal cells are the interop proof (our stack ⇄ OpenSSL).
Done when
http does https:// through the seam with either backend selected; an app
can run one library on OpenSSL and another on native in the same process; the
4-cell client×server interop matrix passes in make lib-test. OpenSSL backend is
the default; native is selectable. Security caveat for the native stack stays
documented ([[feature-tls13-from-scratch]]).
Track B sweep (2026-07-20)
The OpenSSL half is functionally complete through slice 6. The only remaining
umbrella item is the native backend, and that is
[[feature-tls13-from-scratch]]'s deliverable — which the user deferred
("start alongside BSD support, not now") and which has now been moved to
rainy-day/ to match. So a blocked-by edge, not available Track B work: this
ticket cannot close until that one is un-deferred.
Nothing stops someone pulling a single named slice out of the deferred ticket if it becomes urgent (RSA-PSS scheme dispatch, kTLS RX). That does not require un-deferring the umbrella, and it does not change this ticket's state.
Cross-reference to avoid confusion: slice 5 above ("certificate verification +
trust store") is the OpenSSL backend's trust, via
SSL_CTX_load_verify_locations. The separate lib/rtl/truststore.pas landed
2026-07-20 under [[feature-tls-system-trust-store]] anchors the from-scratch
client's chain against /etc/ssl/certs. Two different backends, two different
trust paths; neither supersedes the other.
2026-08-01 (Track B) — measured: the NATIVE backend is the only thing missing
Slice 1 (the seam) shipped. Slice 2, the OpenSSL backend, ships as
lib/rtl/tls_openssl.pas and registers itself. The native backend does not
exist as a unit at all — grep TlsRegisterBackend lib/ finds only
tls_openssl.pas and the two tests.
So the from-scratch TLS 1.3 client, which now does a real
server-authenticated handshake against ed25519, RSA and ECDSA-P256 servers
([[feature-tls13-from-scratch]]), cannot be reached from library code. Its
handshake logic lives in test/devtest_tls13_handshake.pas, a devtest program.
Today an https:// caller's only option is the OpenSSL backend, which needs
dlopen + libssl — and that path is itself blocked on two Track A crashes
(see [[feature-real-dynlib-loader]]). The libc-free client that works is the one
you cannot call.
That is the gap between "the handshake works" and "TLS is usable", and it is this ticket's slice, not the from-scratch ticket's.
What the extraction involves (for whoever takes it)
Not a wiring job — roughly 300 lines of handshake driving move out of the
devtest into a tls13_native.pas implementing TTlsBackend:
Handshake(fd, role, host, var c)— currently a straight-line script over a connected socket; becomes a state machine, or at minimum a blocking call that returnstlsErrorrather than callingFail()and halting.Read/Write— overtls13_record, with the kTLS-TX path chosen as it is today, and the seam'stlsWantRead/tlsWantWritesemantics respected so the async reactor can drive it.Close, and registration ininitialization.
Two properties must survive the move, and both are the kind a refactor drops:
- The fail-closed CertificateVerify dispatch. It rejects any scheme it cannot verify. Until 2026-08-01 that path silently ACCEPTED three of the four schemes we advertise; do not let it regress to a warning.
- Chain verification against the trust store, not just against a CA handed in as a parameter ([[feature-tls-system-trust-store]] is done and has its own devtest).
tools/tls13_handshake_devtest.sh should keep passing UNCHANGED across the
extraction — it is the proof that nothing was lost, so it is worth leaving it
driving the devtest program rather than rewriting it onto the new API in the
same pass.
Slice 2 landed — the NATIVE backend (2026-08-01, Track B)
lib/rtl/tls13_native.pas implements TTlsBackend over the from-scratch TLS
1.3 stack and registers itself. The from-scratch client is now reachable from
library code: TlsHandshake / TlsWrite / TlsRead / TlsClose work with no
libssl and no dlopen.
What moved, and what got better on the way
The handshake came out of test/devtest_tls13_handshake.pas largely as-is, but
three things are deliberately NOT a straight copy, because the devtest's
versions were fine for a loopback test and wrong for a real client:
- Key material is from the OS CSPRNG. The devtest used FIXED bytes — the
private key was
Chr(1)..Chr(32)and the client randomChr(101)..Chr(132). Reproducible is the right call for a test and a catastrophe in a client: a predictable ephemeral key hands the session to anyone who guesses it. NowOSEntropyBytes(getrandom(2)), and failure to get entropy is fatal rather than falling back to a PRNG. - The chain is anchored in the system trust store, via
truststore.VerifyServerChain, instead of against a single CA handed in asParamStr(2). That also means the wholecertificate_listis collected and walked — the devtest parsed only the leaf — so intermediates work, in any order. nowcomes from the clock, not from argv.
Errors return tlsError and set Tls13NativeLastError; nothing calls Halt.
The seam reports one failure value and a TLS failure has many distinct causes,
so "it didn't work" is not a diagnosis.
Limits, stated rather than left to be discovered
- The handshake is blocking, and
HandshakeResumeis a no-op. The seam explicitly permits this ("a backend over a blocking fd simply returns tlsOk/tlsError and never wants"). Driving it from the async reactor needs a real state machine — a separate slice. Faking want-read/want-write would be a lie the reactor would trip over. - Client role only;
tlsServerreturnstlsError. - X25519 + AES-128-GCM / ChaCha20-Poly1305, matching what
BuildClientHellooffers. No HelloRetryRequest, no resumption. - kTLS TX offload when available (AES-GCM only); RX always uses the Pascal record layer.
Also landed: SSL_CERT_FILE
LoadSystemTrust now honours SSL_CERT_FILE before the system bundles — the
convention OpenSSL and curl already use, and the way a caller points at a
private CA. Set-but-unreadable is a hard failure, not a fallback: someone
who named a trust file meant it, and quietly trusting a different set of roots
than the one they asked for is exactly the surprise a trust store must not
spring. It is also what makes the backend testable without weakening the
default.
Proven
tools/tls_native_seam_devtest.sh (make tls-native-seam-devtest), driving a
client that names no tls13_* unit beyond the uses that registers the
backend:
- 3 accepted — RSA (rsa_pss_rsae_sha256), ed25519 and ECDSA-P256 servers,
each a full HTTPS GET returning
HTTP/1.0 200. - 4 refused — untrusted root, hostname mismatch, empty trust file, missing trust file. These carry as much weight as the positives: a client that accepts everything passes every positive test.
Both existing devtests pass unchanged, which is the evidence the extraction
lost nothing: tls13-handshake-devtest (ed25519 + rsa_pss + ecdsa_p256, kTLS-TX
and Pascal fallback) and truststore-devtest (10 assertions, still green after
the SSL_CERT_FILE change).
Registration is EXPLICIT (corrected after review)
The first cut registered the backend from tls13_native's initialization.
That was wrong and was caught by comparing against the only other backend:
tls_openssl registers only when OpenSslTlsRegisterEx is called. Merely
LINKING a unit must not change which TLS stack a program trusts — that is a
process-global, and deciding it by link order is exactly the kind of thing that
is invisible until it bites.
So Tls13NativeRegister is an explicit call and the initialization section
registers nothing. That also happens to be the mechanism the "which backend
when both are present" question needs: calling one registrar after the other is
a deterministic override rather than a race between two initialization
sections. The seam devtest asserts not TlsAvailable before it registers, so
the absence of the side effect is tested rather than assumed.
What this unblocks
https:// over the native stack no longer needs [[feature-real-dynlib-loader]],
which is blocked on two Track A crashes. The remaining work to make it the
default for http.pas is choosing a backend when both are registered — not
covered here.
End to end: HttpGet over https:// with no OpenSSL in the process
http.pas already routed https through the seam and already treated the
handshake as completing within one TlsHandshake call — which is exactly what
this backend does — so nothing there needed changing. Verified rather than
assumed:
backend=native-tls13
status=200 reason=ok
HTTPS OK
Asserted in the seam devtest as HttpGet over https (no libssl in the process), driven by test/devtest_https_native.pas, which names no TLS unit
beyond the registrar.
Two stale claims in http.pas's header fixed while there: it said "no backend
ships in-tree yet" (two do now, and it now names both registrars and what each
costs), and that only "the mock and a blocking OpenSSL backend" complete the
handshake in one call.
Slice 3 is the ASYNC handshake — user-prioritised (2026-08-01)
Rene, asked which of the remaining TLS items matters: "async is a primary feature." So this is not a nice-to-have behind interop breadth — it is the next slice, and it is pre-approved: a fresh session should start it without re-asking for scope.
Today Handshake is blocking and HandshakeResume is a no-op. The seam permits
that and http.pas is happy with it, but the coroutine reactor cannot drive it,
so every https request occupies a thread for the length of a handshake. Making
it real means a state machine over the flight parser — the loop that currently
blocks in ReadRecord becomes a resumable step returning tlsWantRead — and
HandshakeResume picking up where it left off.
Do NOT fake it: returning a want the backend cannot honour is worse than
blocking, because the reactor will believe it. The two properties in the header
above (fail-closed CertificateVerify, trust-store anchoring) must survive the
rewrite, and tls13-handshake-devtest + truststore-devtest must keep passing
unchanged as the evidence they did.
2026-09-01 (frankH) — the blocked-by was HIDING the slice the owner asked for
This ticket has been suppressed from the Track B ready queue, and the thing it was hiding is slice 3 — the one Rene named directly: "async is a primary feature", recorded above as pre-approved, "a fresh session should start it without re-asking for scope."
The edge said blocked-by: [feature-tls13-from-scratch]. That ticket is
deliberately parked in rainy-day/ (Track B sweep, 2026-07-20) because its
own status line has read "DEFERRED, start alongside BSD support (not now)"
since it was opened. ready_tickets keeps a ticket whose blockers are all
resolved — parked is not resolved — so progress.sh ready --track B did not
list this one at all. Checked, not inferred: it was absent from that queue
while sitting at p53.
The edge was false, not merely stale. The parking note says it outright:
"Nothing is lost: M1-M7 are implemented and tested... The next slices can be
pulled individually if one becomes urgent — that does not require un-deferring
the umbrella." And this ticket's own body already records the native backend
landing (backend=native-tls13, status=200, HTTPS OK).
Verified by running it rather than by reading either ticket —
make tls13-handshake-devtest:
tls13-handshake-devtest OK (ed25519 + rsa_pss + ecdsa_p256; chain verify;
kTLS-TX + Pascal fallback)
Three signature schemes, chain verification against the trust store, a real https GET through the Pascal record layer. Nothing here is waiting on the deferred umbrella.
Shape of the failure, because it is the third of this kind today. A
blocked-by edge is a claim about the world at filing time and nothing
re-checks it; resolving or PARKING a blocker is an event on the blocker, and
the edge lives on the dependent, so at the moment the claim goes wrong nobody
is standing where it is written. Here it cost more than tidiness: the queue
stopped offering work the owner had personally prioritised, and an absence is
not something anyone notices.
2026-09-01 (frankH) — slice 3: async https WORKED ON, and it was BROKEN, not slow
Track B. The slice was pre-approved as "async is a primary feature". What I found first changes the framing, so it goes first.
The premise was wrong, and the truth was worse
This ticket said "every https request occupies a thread for the length of a handshake." It did not. Async https did not work at all. Same server, same URL, one binary, both paths:
SYNC ok status=200
ASYNC FAILED lasterror=[no ServerHello (connection closed)]
The async path makes the fd non-blocking before the handshake runs
(TcpConnectAddr -> PalSetSocketNonBlocking), and RecvN tested
if got <= 0 — which swallows PAL_NET_EAGAIN along with a real EOF. So a
would-block was reported as a closed connection, about a connection that was
open and healthy. That is why the message was never a lead: it accused the
peer.
SendBytes had the twin defect and was quieter still: one NetSend, result
ignored — EAGAIN sent nothing, a short write sent a truncated record, and both
corrupt the stream rather than failing.
No state machine — because the scheduler is STACKFUL
This ticket prescribed "a state machine over the flight parser". I did not
build one, and the reason is a measurement rather than a preference:
scheduler.WaitIO ends in __pxxcoswitch, a real stack switch, and each
coroutine owns a heap stack. So a yield from six frames down inside the flight
parser resumes exactly where it left off with every local intact. The stack IS
the state machine, and it is already written and already tested.
So the fix is one pair of EAGAIN-aware helpers used by the handshake AND the
record layer, with WaitForFd choosing how to wait:
- inside a coroutine ->
WaitReadable/WaitWritable: park on epoll and yield, thread free. This is the async property the ticket wanted. - outside one ->
PalPoll: block. No reactor to stall.
It asks the new scheduler.InCoroutine rather than threading an async flag
through Handshake/Read/Write, because the seam's signatures do not carry
one and widening them would push the transport choice onto every caller of a
deliberately backend-neutral API.
This is not the thing the ticket warned against. "Do NOT fake it:
returning a want the backend cannot honour is worse than blocking." Nothing
here returns a want it cannot honour — it returns no wants at all, which the
seam explicitly permits, and it genuinely does not occupy the thread.
HandshakeResume stays a legitimate no-op.
InCoroutine is deliberately non-attaching: CurR attaches a reactor slot
on first use, so a predicate built on it would consume one of 64 slots merely by
being asked, from threads that never run a coroutine.
The second defect, found because the first was fixed
With the I/O fixed, async https segfaulted. The default coroutine stack was
64 KB and a handshake does not fit. Measured rather than guessed — 64 KB and
96 KB crash, 128 KB up passes — so CO_STK is now 192 KB, the measured floor
plus 50%. It is charged per LIVE coroutine, not per slot, and constrained
devices already call SpawnSized explicitly, so they are unaffected.
The guard that could not fire for its own case
The canary check sat inside the coState = 2 (done) arm, so a coroutine that
overflowed and died before finishing was never checked — which is every
serious overflow. Moved to every return to the scheduler.
Both directions measured, on the same binary and the same input — a coroutine that clobbers the canary, yields, then dies:
| check site | result |
|---|---|
| old (completion only) | silent SIGSEGV, no output, rc 139 |
| new (every yield) | fatal: coroutine stack overflow (canary clobbered), rc 217 |
And the honest limit, which is in the source comment too: it still does NOT catch an overflow that faults immediately, before it can yield. The 64 KB TLS case is exactly that and still segfaults silently. This narrows the window; it does not close it. A guard page per stack would, at an mmap each.
Evidence
test/devtest_https_native_async.pas + a row in
tools/tls_native_seam_devtest.sh. Plain Spawn, not SpawnSized, so it also
asserts the default stack suffices.
The control that matters: with the EAGAIN fix reverted, the SYNC row still
passes and only the async row goes red. That is the whole argument for the new
row existing — devtest_https_native passed throughout the period when every
async https request failed.
tls13-handshake-devtest |
OK (ed25519 + rsa_pss + ecdsa_p256, chain verify, kTLS-TX + Pascal fallback) |
tls-native-seam-devtest |
OK (3 schemes, 4 refusals, https via http.pas, sync + async) |
gate.sh quick |
GREEN |
| lib suites | lib_tls 16, lib_http 83, lib_http_async 5, lib_asyncnet6, lib_tls13_{keys 5, record 6, hs 6} — all at the counts the Makefile asserts |
| scheduler | SCHED WIDE OK; exhaustion arm rc 216 with its exact fatal |
The four refusals passing is the part that matters most: fail-closed CertificateVerify and trust-store anchoring both survived.
Two scope notes. The FPC seed canary SKIPPED, correctly — this change is
lib/rtl only and touches no compiler/**, so it carries no FPC-seed risk.
And make lib-test cannot currently run to completion for anyone: its first
check, crtl_reachability, fails on <string.h> declaring strsignal()
defined in signal.c. Verified pre-existing at a clean HEAD with my changes
stashed; it is Track C's and is routed to frankC.
Still open on this ticket
Application-data Read/Write now inherit the EAGAIN handling through the
shared helpers, but they are only exercised by the devtest's single GET. A
large streamed body over the reactor is untested. Server role is still refused.
2026-09-05 (frankH) — staleness check: the routed blocker is gone, and the frontmatter was the defect
Three things, none of them new code.
1. The routed Track C blocker is GONE — and make lib-test still does not
finish, for a different and temporary reason. The section above says the suite
"cannot currently run to completion for anyone", blocked on
crtl_reachability failing over <string.h> declaring strsignal(), and routes
it to Track C. Re-measured today: crtl-reachability: OK -- 148 headers, 66 modules, every declared function reachable from its own header, and the run goes
straight past it, through crtl-map: OK -- 608 crtl functions, and stops two
steps later at lib-units: FAIL mimic_string / mimic_urllib_request on
undefined variable (pyvar_is_objtag). That is the fleet-wide pin lag, not
this ticket's problem: frankZ exported pyvar_is_inttag/pyvar_is_objtag from
compiler/builtin/pylib.pas, lib/rtl calls them, and $(PXX_STABLE) is the
pin, which has not caught up. The remedy is a pin and it is with the owner.
A routed blocker is the shape most likely to go stale unnoticed — the person
who fixes it is not standing where the claim is written, which is the same
asymmetry the 2026-09-01 blocked-by note describes one section up. Nobody told
this ticket, and nobody would have. Note also that "still red" would have been
the wrong reading twice over: the red moved from a real Track C defect to a
temporary pin gap, and only running it says which.
2. The frontmatter was four lines and none of them said what this is. No
status, no owner, no type, and no summary at all, on a p53 ticket
sitting in working/. owner: frankH and Status: working existed only in the
prose header. Anyone reading the board saw a title and a number. Fixed, with a
summary that leads with what actually works, because a reader deciding whether
to take this needs to know that https works today through two backends before
they need anything else.
3. The three open items, re-stated so they are findable. A large streamed
body over the reactor is still untested — the devtest does a single GET, so the
EAGAIN handling the application-data path inherited from the shared helpers is
exercised once and shallowly. The native backend is client-role only
(tls13_native.pas:295); the OpenSSL backend has the server side. And the
coroutine stack canary narrows the overflow window rather than closing it: it
still cannot catch a fault that happens before the first yield, which is exactly
the 64 KB TLS case.
2026-09-05 (frankH) — the streamed body is covered now, and the code was already right
The first of those three is closed. test/devtest_https_native_stream.pas plus
three rows in tools/tls_native_seam_devtest.sh fetch a 2 MB body over
https on both paths and assert its LENGTH and a byte SUM.
The gap was in the testing, not in the code. Both paths return
len=2000000 and the exact sum of the served file, first run, no fix needed.
That is the honest result and it is still worth having: the row that existed
fetched openssl's status page, which can complete without a single would-block,
so it could not fail the way the handshake failed for months — a short read
taken as end-of-stream. In the data path that same shape truncates a body, and a
truncated body still parses, still says 200, and still looks like a success.
Both quantities, because neither subsumes the other. Truncation moves the length; a dropped or reordered record in the middle keeps the length and moves the sum.
The expectation is derived from the served file on every run (wc -c and an
od/awk byte sum), not written down, so there is no constant to keep in step
with a fixture. The body is built by repeating a file already in the tree, which
is deterministic and varied and needs nothing a POSIX shell lacks.
And there is a control on the control. The third row fetches a SMALL file with the big file's expectations and asserts the comparison REJECTS it. A length-and-sum check that is silently comparing nothing passes every positive row, and that is a guard which cannot fail. It rejects, so it is live.
Harness note worth keeping: openssl s_server -HTTP answered nothing at all
here — verified with curl, code=000 — while -WWW served the full 2 MB
(code=200 size=2000000). The existing rows use -www, which serves a status
page rather than files, so this is a third mode and not a typo.
One thing I nearly shipped, recorded because the near-miss is the lesson.
The first draft routed the response through a local with a comment saying pxx
refuses a function result as a const record argument and fpc accepts it. That
was false. The original error was a cascade from HttpRequest not being
exported from http.pas — only HttpGet and HttpRequestAsync are — and the
no overload of Measure matches line was about the unresolvable inner call, not
about the record. Probed both spellings against pxx and fpc before believing it;
both compile. The comment and the local are gone. A source comment asserting a
compiler defect that does not exist is worse than no comment, and I had already
written the sentence that would have gone into a ticket.
Still open: the native backend's server role, and the stack canary's before-first-yield window.