Networking runtime

• Type: feature
• Status: backlog
• Owner: —
• Opened: 2026-06-07 (manual request, consolidates plan-networking.md)

Summary

Add a target-neutral networking API (net.pas) with platform-specific backends. Use the Synapse library as both a compiler compatibility target and correctness test suite.

Design Reference

Full design lives in devdocs/developer/plan-networking.md. Key decisions:

• Public API: TNetSocket, TNetAddress, Connect, Listen, Accept, Send/Recv, Close. Later: Poll, TLS hooks, async integration.
• Three backends:
  • net_linux_sys.pas — raw Linux x86-64 syscalls, no libc.
  • net_posix.pas — libc sockets + getaddrinfo.
  • net_esp32.pas — ESP-IDF/lwIP sockets.
• Abstraction strategy: syscall preferred on Linux, library on ESP32. Backend selection at compile time via target conditionals.

Analysis: Work Required

1. Compiler prerequisites (may already be in progress)

2. Runtime implementation order

1. Syscall constants + sockaddr_in struct layout for Linux x86-64.
2. Byte-order helpers (htons, ntohs, htonl, ntohl).
3. Thin syscall wrappers: socket, bind, listen, accept, connect, read/write, sendto/recvfrom, close, setsockopt.
4. Public net.pas API wrapping the above.
5. Loopback tests: TCP echo client/server, UDP send/recv.

3. Synapse compatibility milestones

First pass: classify failures. Second pass: fix compiler blockers. Third pass: compile units successfully.

4. Test strategy

• Loopback-only tests (no external network dependency).
• Synapse smoke scripts already exist: test/manual/try_synapse_compile.sh.
• Add automated make test targets once basic compilation works.

First Milestone

Linux syscall-only IPv4:

• TCP client with IP literal.
• TCP server with SO_REUSEADDR, bind, listen, accept.
• UDP sendto / recvfrom.
• Loopback tests only.

DNS deferred to feature-dns-resolver-library.

Log

• 2026-06-07 — ticket opened; consolidated from user note and plan-networking.md.

• 2026-06-10 — relative-path units delivered (4aa293a); improved uses error now shows the synsock failure is a missing syncobjs RTL unit, i.e. RTL availability, not path resolution. Other three smoke units still fail on conditional-directive parse.

• 2026-06-10 — quick-win pass (440a9e0, f81ea83): syncobjs RTL stub added; directive layer now digests jedi.inc fully (inactive-branch {$IF} eval skipped per FPC semantics, {$IFOPT} recognized, {$IFEND} accepted, define table 128→1024). All four smoke units now fail on ONE remaining blocker class: platform-branch selection. PXX predefines LINUX but not FPC, so jedi.inc picks the Kylix path (uses libc/system). Predefining FPC globally would break self-host (compiler source uses {$ifdef FPC} to mean "real FPC, not PXX") — needs a design decision (per-source define set, or a PXX-aware branch in install step, or -d FPC only for foreign code). After that: RTL availability (synafpc, termio, sockets, netdb, Classes surface).

• 2026-06-10 — platform-branch decision made: opt-in mimic mode, see feature-mimic-fpc (35345a3). Synapse compatibility milestones wait on it; the syscall net.pas milestone does not.

• 2026-06-16 — the syscall net.pas milestone has a concrete start: lib/rtl/asyncnet.pas — an async TCP socket layer (TcpListen/Accept/Connect/Recv/Send/Close) over the coroutine epoll reactor (feature-async-coroutines), x86-64, raw Linux syscalls, no libc. Proven by test/test_asyncecho.pas (concurrent echo server). Not yet the target-neutral TNetSocket/TNetAddress API or the cross/esp32 backends; asyncnet is the Linux-x86-64 async backend the abstraction will wrap. Cross-target parity tracked in feature-cross-target-feature-parity.

• 2026-06-19 — strategy locked twofold (see plan-networking.md "Strategy" section). (1) Own async/syscall/ESP socket layer — the asyncnet.pas reactor is its Linux-x86-64 start; build the target-neutral TNetSocket/TNetAddress

  • cross/esp32 backends independently of Synapse. (2) Compile Synapse via its Delphi-Posix.* path (not the FPC/BaseUnix path) to reuse its HTTP/FTP/ SMTP/POP3/DNS clients — yields BLOCKING clients (welded to TBlockSocket), good for one-shot client tasks, not async. Build one transport core, expose two faces (native async API + Posix.* compat shim). The Posix.* shim is 6 thin units over our syscalls (SysSocket/SysSelect/SysTime/StrOpts/Errno + pure-data NetinetIn) — syscall-only achievable, no libc. DNS is not libc-blocked: reuse Synapse synadns (UDP/RFC1035) over our UDP syscalls, nameserver from /etc/resolv.conf. Synapse goal depends on feature-mimic-fpc (curated define profile to select Posix.*) + {$mode delphi} handling (mainly relax @ to untyped). SSL deferred (pluggable ssl_openssl, blocking ok).

• 2026-06-19 — dual-facade decision. Expose BOTH naming layers as thin facades over one private syscall transport core (siblings, neither calls the other): (2) FPC-native BaseUnix/Sockets/UnixType — built anyway for own-RTL + compile-FPC-source, and FPC's own versions are already syscall-based; (3) Posix.* — the clear C-header surface (libc-backed on real Delphi, syscall-backed here). The scoped manifest picks which branch each library compiles against → reach compat "left or right". If ever collapsing to one master, FPC-native is master (Posix.* wraps it). ESP caveat: Posix.* is Unix-only (no ESP); the portable layer is TNetSocket, with Posix.*/FPC-native as Unix porting facades. DNS: FPC netdb (pure-Pascal resolver, syscall-shaped) joins synadns as a reference. Linux net config via /proc+/sys reads (no libc/ioctl). FPC RTL itself is largely libc-free — validates the syscall model.

• 2026-06-19 — layering refined: Posix.* is the canonical base; FPC wraps it. Posix.* base API has a SELECTABLE backend — posix_syscall (default, "just works") and posix_libc (opt-in via define PXX_POSIX_LIBC), same interface so the user picks syscall vs libc. Cost is ~1.3x not 2x: types/structs shared in one include (NetinetIn is pure data), only function bodies differ (syscall = the meaty impl we want; libc = trivial externs). FPC-named units (BaseUnix/Sockets/UnixType) are thin wrappers OVER Posix.* (master question settled: Posix is master). ESP seam softened: lwIP's BSD-socket API is Posix-shaped, so a 3rd backend posix_lwip lets Posix.*/FPC reach ESP with documented gaps — "backend differs", not "API absent". Portable cross surface stays TNetSocket; async stays the epoll reactor (Posix/FPC are blocking compat surfaces, coexist not merge).

• 2026-06-21 — PAL network foundation landed under feature-platform-abstraction-layer: IPv4 TCP primitives in platform.pas with POSIX raw-syscall backend and ESP-IDF/lwIP object-shape backend. Regression test/lib_platform_net.pas proves POSIX loopback TCP; native --platform=esp tests assert no host fallback; C3/S3 object smokes import the expected lwip_* symbols. Remaining PAL networking gaps (UDP, poll/select readiness, exact errno, ESP-IDF run validation, IPv6/DNS hooks) are split to feature-pal-network-datagram-poll-errno.

• 2026-06-21 — Blocking/async layering decision documented in devdocs/developer/plan-networking.md: keep one PAL socket substrate, then expose two top-level libraries above it. net.pas is the normal blocking API; asyncnet.pas is coroutine-backed and viral by design. PAL must stay scheduler- free and provide only socket ops, nonblocking mode, readiness/error primitives, and portable capabilities.

• 2026-06-21 — Synapse support audit continued against the pinned stable compiler with the manual smoke helper. test/manual/try_synapse_compile.sh now defaults to stable_linux_amd64/default/pinned and has SYNAPSE_PROFILE=posix as a temporary stand-in for the future scoped manifest. Default profile still falls into the wrong/missing branch (libc/system). The POSIX profile exposes two compiler blockers before the library/PAL shims can matter: dotted namespace unit names (Posix.*, System.Generics.*) and {$IF DECLARED(Qualified.Symbol)}. Filed feature-dotted-unit-names and feature-conditional-declared-directive; no Synapse source workaround.

• 2026-06-21 — DNS resolver policy split out to feature-dns-resolver-library: dns.pas facade with selectable dns_libc (getaddrinfo), dns_wire (pure Pascal over PAL UDP/TCP), and dns_resolved (systemd-resolved over D-Bus) backends. Public DNS fallback is explicit opt-in only, never default.

• 2026-06-22 — PAL socket substrate is now complete enough for the blocking net.pas first milestone (Track B, stable v37). On top of the TCP primitives it now provides: UDP PalSendToIpv4/PalRecvFromIpv4 with peer reporting, PalPoll readiness (raw ppoll), -errno returns plus PAL_NET_EWOULDBLOCK/ECONNREFUSED/ECONNRESET (feature-pal-network-datagram-poll-errno, done), and socket introspection PalGetSockError (SO_ERROR), PalGetSockNameIpv4, peer-reporting PalAcceptIpv4 (commit adebdf9). All host-proven on loopback via lib_platform_net* in make lib-test. The PAL stays scheduler-free; remaining net.pas work is the target-neutral TNetSocket/TNetAddress blocking API over these primitives — no new PAL surface expected for IPv4 loopback TCP/UDP. Still PAL-blocked above IPv4: IPv6 sockaddr layout (PAL_NET_AF_INET6 + 28-byte sockaddr_in6 fill/parse) — to be added when net.pas reaches it.

• 2026-06-22 — First-milestone lib/rtl/net.pas landed (Track B, stable v37, commit 3d7ac46): the blocking IPv4 face the milestone called for, with no platform conditionals of its own. TNetSocket/TNetAddress; TCP NetTcpListen/NetTcpAccept (peer-reporting)/NetTcpConnect/NetSend/ NetRecv; UDP NetUdpBind/NetUdpSendTo/NetUdpRecvFrom; plus NetGetSockName/NetGetSockError/NetShutdown/NetClose. End-to-end loopback proof in test/lib_net.pas (single thread: blocking connect completes via the kernel backlog, then accept; TCP echo + UDP roundtrip + ephemeral bind/getsockname + peer address) wired into make lib-test. asyncnet.pas stays the coroutine face over the same PAL primitives. STILL OPEN under this ticket: IPv6, DNS (feature-dns-resolver-library), the Synapse / Posix.* compat path (Track A blockers feature-dotted-unit-names + feature-conditional-declared-directive), and async/blocking facade unification.

• 2026-07-19 (backlog sweep note) Milestone note: lib_synapse smoke in make lib-test compiles real Synapse under --mimic-fpc and runs b64/md5/sha1/crc32 + TCP ping/pong — the Synapse milestone is effectively achieved (see resolved feature-synapse-compile-check). Remaining here: IPv6, async/blocking facade unification; DNS split to its own ticket.

Log

• 2026-07-20 (Track B) — IPv6 reaches the PAL. TPalIn6Addr (16 wire-order bytes) in platform_types.pas; PalBindIpv6 / PalConnectIpv6 / PalIn6Loopback / PalIn6Any on the facade; a real posix backend building sockaddr_in6 (28 bytes: family, port, flowinfo, 16 address bytes, scope id). Gated by test/lib_ipv6.pas in make lib-test — binds ::, connects to ::1, moves a payload.

  Two deliberate choices worth keeping:

  • TPalIn6Addr is a byte array, not four LongWords. An IPv6 address is a byte string on the wire; every LongWord view invites a byte-swap that must not happen. The IPv4 path takes a host-order LongWord and swaps; the v6 path copies. Conflating the two is the classic way to ship an address that looks right in a debugger and is wrong on the wire.
  • The ESP backend returns PAL_ERR_UNSUPPORTED rather than attempting it. lwIP can do IPv6, but only when the IDF build has LWIP_IPV6 on, and the backend cannot ask. Emitting a sockaddr_in6 at a v4-only lwIP would fail with a confusing errno; refusing honestly is better until someone runs it on a device with IPv6 enabled.

  The test SKIPs (exit 0) when socket(AF_INET6) fails, since a host with IPv6 disabled in the kernel is not a code defect. What it is really guarding is the sockaddr_in6 byte layout — get a field offset wrong and bind fails or, worse, silently binds the wrong address.

  Not done: scopeId is plumbed through but untested — a link-local (fe80::/10) connection is the only thing that exercises it and that needs a real interface index. TNetAddress / net.pas and asyncnet still speak IPv4 only; surfacing v6 there (and dual-stack listeners, and AAAA in the resolver) is the next slice. So this is the PAL floor, not IPv6 support in the networking library.

• 2026-07-20 (Track B) — IPv6 surfaced in net.pas. TNetAddress gained Family, V6 and ScopeId; NetAddress6 / NetLoopback6 / NetAny6 / NetIsV6 / NetTcpAccept6 added; NetTcpListen and NetTcpConnect now branch on Family. Gated by test/lib_net6.pas (v6 round trip and proof the v4 path is unchanged — with a new field in a shared record, "does v4 still work" is the more important half).

  Family is set by the constructors, never left to chance. A TNetAddress built by hand has an indeterminate Family, and that field now decides which socket family gets created — so NetTcpAccept/NetUdpRecvFrom initialise the peer they fill in, and the record's declaration says to construct through the helpers. Only two sites in the tree built one by hand, both in tests.

  Known gap, stated rather than hidden: NetTcpAccept6 does not fill in the peer address, only its Family. There is no PalAcceptIpv6 yet — the PAL's accept fills an IPv4 sockaddr — so reporting a zeroed v6 peer as if it were real would be worse than reporting nothing. Adding PalAcceptIpv6 is the next slice.

  Also still IPv4-only: UDP (NetUdpBind/SendTo/RecvFrom), asyncnet, and AAAA lookups in the resolver. Dual-stack listeners (IPV6_V6ONLY) are untouched. So TCP client and server now speak IPv6; the rest of the library does not yet.

Umbrella status (2026-07-20, Track B sweep)

This is a strategy umbrella spanning years of milestones (own syscall socket layer, PAL backends, Synapse compatibility via the Posix.* path, cross/ESP targets), not a unit of work that can be finished in one pass. Its named milestones are done — net.pas, asyncnet, the PAL, and per the 2026-07-19 sweep note the Synapse milestone is "effectively achieved".

Its concrete open remainder is now [[feature-ipv6-complete-surface]] (accept peer address, UDP v6, asyncnet v6, AAAA, dual-stack, scopeId). Prio dropped to 20 to reflect that: rank the children, not the umbrella, so it stops presenting as available work while its actual tasks live elsewhere.

2026-07-31 — the IPv6 child is closed

[[feature-ipv6-complete-surface]] is resolved: v6 accept returns a real peer (address, port and scope id), UDP branches on family through both the PAL and net.pas, and asyncnet gained TcpListen6 / TcpConnect6 / TcpConnectAddr6. Gated by lib_net6 (extended) and the new lib_asyncnet6, both SKIP-clean on a host without AF_INET6.

Two items came back out of it as a Track U question rather than plumbing — IPV6_V6ONLY on a :: listener, and A-vs-AAAA ordering — filed as [[decide-ipv6-dualstack-and-aaaa-ordering]]. Both are choices this umbrella's design section never made, and both change observable behaviour, so they wait for a human.

2026-08-02 (Track B) — the decided items landed; this umbrella has no open children

[[decide-ipv6-dualstack-and-aaaa-ordering]] was resolved 2026-08-01, but the ticket that owned its implementation ([[feature-ipv6-complete-surface]]) was already in done/, so the decided work sat in nobody's queue for a day. Both halves are now in:

• IPV6_V6ONLY escape hatch — NetTcpListen(..., v6Only) with INHERIT (the decided default, unchanged) / OFF / ON. Asserted behaviourally: strict refuses a v4 client, dual-stack accepts one. Commit 272a4d90b.
• A-first, AAAA-fallback connect-by-name — new lib/rtl/netconnect.pas, kept out of both net.pas and dns.pas so neither gains a dependency on the other. Commit ec221f7a0, re-filed as [[feature-net-a-first-connect-by-name]].

Two defects fell out of that work, both fixed with regressions: [[bug-b-dns-wire-ipv4-literal-returns-nxdomain]] (the default resolver answered NXDOMAIN for 127.0.0.1, so the facade's answer changed with the backend), and NetTcpConnectTimeout silently ignoring addr.Family (a v6 address connected to 0.0.0.0).

State of this umbrella: no open children. Its named milestones are done, its IPv6 child is closed, and its decision is implemented. The only networking tickets left in the tree are [[feature-dns-esp-backend]] (ESP-only, and not startable without an ESP runner) and [[feature-dns-libc-backend]] (done).

It is still ranked 20 and still appears in the ready queue despite its own instruction to "rank the children, not the umbrella". Whether a years-spanning strategy umbrella with zero open children should be closed, or kept open to collect future ones, is a judgment call about how this project wants its board read — flagged here rather than decided by an agent.