FreeBSD and the BSDs
PXX writes native FreeBSD/amd64 and FreeBSD/arm64 executables from Pascal, C and Nil Python. Every kernel but Linux is EXPERIMENTAL, FreeBSD included: the label stays until PXX programs are in daily use on live systems across several test machines (GUI, userland, advanced networking). FreeBSD is the most complete of them; OpenBSD, NetBSD, DragonFly BSD and Minix have their notes at the end of this page.
./pxx --os=freebsd hello.pas hello # copy hello to the FreeBSD box and run it
./pxx --os=freebsd --target=aarch64 hello.pas hello # the same for FreeBSD/arm64
Measured on FreeBSD 15.1 (November 2026). FreeBSD 14 or newer is needed:
timerfd arrived in 14.
How it works
A FreeBSD build is a Linux build with one extra piece. The program still needs
no libc, and the runtime libraries are the same ones a Linux program uses. The
difference is the kernel entry: every system call goes to a small translator,
compiler/builtin/fbsdabi.pas, that is compiled into the program. It
translates in both directions:
- system call numbers
- flag bits and constants (
openflags,mmapflags, socket options,ioctlrequests and so on) - struct layouts (
stat,sockaddr,msghdr,termios,sigaction,siginfo,ucontextand more) - signal numbers
errnovalues
Anything the translator does not recognise returns ENOSYS, as an unknown
system call does on Linux.
Some Linux interfaces have no FreeBSD system call of their own. The translator builds them from FreeBSD parts:
| Linux | Built from, on FreeBSD |
|---|---|
epoll |
kqueue |
futex |
_umtx_op |
clone (threads) |
thr_new, in the compiler's clone stub |
/dev/ptmx, TIOCGPTN |
posix_openpt, FIODGNAME |
fallocate (mode 0) |
posix_fallocate |
personality |
a value kept in the program |
prctl(PR_SET_NAME / PR_GET_NAME) |
thr_set_name and a kept copy |
/proc/self/environ, /proc/self/exe, /proc/self/fd/N, /proc/sys/kernel/hostname |
the startup block, sysctl, fcntl(F_KINFO) |
/proc/self/fdinfo/N, /proc/mounts (and /proc/self/mounts), /proc/stat |
fstat, getfsstat, kern.cp_time(s) (see the /proc files) |
The executable carries the NT_FREEBSD_ABI_TAG note every native FreeBSD
program has. Without it, FreeBSD reports a stack overflow as SIGBUS instead
of SIGSEGV.
What works
Each item below has been run in a FreeBSD VM and compared with the Linux build of the same program: the output has to be identical.
- files, directories,
stat, links,statfs - environment, arguments, the program's own path
- processes:
fork,exec,wait, pipes,subprocess - signals, including a crash turned into an exception
select,polland the async scheduler (epollon top ofkqueue)- timers (
timerfd) andeventfd - sockets: IPv4, IPv6 and Unix, including
sendmsg/recvmsgand passing file descriptors (SCM_RIGHTS) - terminals (
termios) and ptys - threads built with
--threadsafe:TThread, Cpthread_create,threadvar,thread_local, mutexes, parallel for - the scheduler calls,
mlock,fallocate,swapon/swapoff - System V IPC (
shmget/shmat,semget/semop/semctl,msgget/msgsnd/msgrcv, and the three*ctlcalls), on all four BSDs. The*_dsstructs come back in Linux's layout. A few fields are what the system itself says: FreeBSD roundsshm_segszup to a page, NetBSD leavesmsg_cbytesat 0, and DragonFly leavessem_otimeat 0. Linux-only commands (IPC_INFO,SHM_STAT,MSG_EXCEPTand the like) sayEINVAL. Only NetBSD hassemtimedop; elsewhere a timed wait retriessemopevery millisecond until it succeeds or the time runs out (EAGAIN).
Every Pascal library test (test/lib_*.pas, 157) and C test (test/c_*.c,
224) was run on both kernels on 2026-11-12. Every difference that was a
translator bug was fixed. The rest are listed under
Not supported or were caused by the test environment.
FreeBSD/arm64
--os=freebsd --target=aarch64 uses the same translator. On arm64 the
runtime speaks Linux's generic system call table, so the translator has a
front layer (compiler/builtin/lxa64front.inc, shared with NetBSD) that maps those numbers
onto the x86-64 ones the rest of it understands and converts the four things
that differ between Linux's two 64-bit ABIs:
struct stat(128 bytes on arm64;st_nlinkandst_modediffer in size and place);struct epoll_event(16 bytes, not packed);struct sigaction(nosa_restorer, so the mask is at offset 16);- four
openflags (O_DIRECTORY,O_NOFOLLOW,O_DIRECT,O_LARGEFILEhave other values).
arm64 has no fork call: clone(SIGCHLD) becomes FreeBSD's fork.
The compiler side: every svc #0 the aarch64 backend emits becomes a bl to
one shared stub that calls the translator, so no hand-written branch distance
moves. FreeBSD/arm64 starts a program with the argument block in x0 and sp
rounded down to 16 bytes; the first instructions slide argc, argv, envp
and the auxiliary vector down so that sp points at argc, as on Linux.
Threads start with thr_new, which is handed the new thread's TLS block as
its tls_base (FreeBSD does not copy tpidr_el0 to a new thread). Signal
handlers get a Linux arm64 ucontext, so __pxxSigPCPtr works.
make test-freebsd-arm64 runs about 70 programs on an arm64 VM and compares
them with the Linux aarch64 build run under qemu-aarch64: files, signals,
threads, sockets, async I/O, timers, termios on a pty, ptys, -g, and the C
and NilPy runtimes. It passes. The VM runs under TCG emulation, so it is slow:
PXX_FBSD_ARCH=aarch64 tools/vm/freebsd.sh start, port 2228, image under
/data/vm/freebsd-arm64. Its first boot installs base-system updates and
takes about 20 minutes.
Not supported
The compiler refuses these with a message that names the reason:
- CPUs other than x86-64 and aarch64 (
--target=arm32and so on) --sharedon aarch64: a library is x86-64 only, on Linux too.--emit-objon NetBSD/arm64 and OpenBSD/arm64. The aarch64 backend reaches its data through 32-bit absolute addresses in the text. NetBSD/arm64 links programs above 4 GiB, where those cannot reach, and OpenBSD/arm64 links PIE, which refuses them (with-nopieit links and the program crashes; not looked into). FreeBSD/arm64 works (below).--rtl-libc
These compile, but the call fails when the program runs:
- Threads through glibc's
pthread_create, the route a program takes when it links the system C library. Use--threadsafethreads. - Linux-only
/procfiles. Only the ones listed in the table above exist;/proc/self/maps,/proc/self/statand the rest do not. - Linux-only interfaces:
SCM_CREDENTIALS, abstract Unix sockets,inotify, extended attributes (xattr),klogctl,adjtimex, Linux-specific deviceioctls. acctis left unmapped on purpose. Tests in the VM run as root, and a workingacctcall there would switch on process accounting.- A raw
syscallinstruction in inline assembly goes straight to the FreeBSD kernel and never reaches the translator, so it is read with FreeBSD's numbers. For example, 39 isgetpidon Linux andgetppidon FreeBSD. Call the C library function instead. - A BSD
ioctlnumber written by hand. Everything else in the program goes through the translator, and the translator expects Linux's numbers: use the constants from pxx's<sys/ioctl.h>and<termios.h>. FreeBSD's ownTIOCSCTTY(0x20007461) is unknown to the translator and answersENOTTY; Linux's (0x540E), from the header, works. This applies to all four BSDs.
Device numbers are real FreeBSD numbers: /dev/null is 0:20, not Linux's 1:3.
Objects and shared libraries
On x86-64, --emit-obj and --shared work for FreeBSD, NetBSD, DragonFly
and OpenBSD, and the system's own cc links the result into a C program.
On FreeBSD/arm64, --emit-obj works too (since 2026-11-15), and
make test-freebsd-arm64 links the object half of the test below:
./pxx --os=freebsd --emit-obj mylib.pas mylib.o # then, on FreeBSD:
cc main.c mylib.o -o prog
./pxx --os=netbsd --shared mylib.pas libmylib.so # cc main.c -L. -lmylib
The export surface and the rules are the Linux ones (see
Linking a pxx object into another program): the
C-convention routines are exported, and the program body runs from the
host's init array before main. The translator is ordinary code inside the
object, so the pxx half makes its own system calls while the host's half
uses the system's libc. Two things are specific to these systems:
- A library's text is mapped read+execute and its data read-write, in two segments. NetBSD (PaX MPROTECT) refuses to map one writable, executable segment, and OpenBSD enforces W^X.
- On OpenBSD only the default (libc) build can be an object or a library.
--rtl-rawis refused there, because OpenBSD accepts a system call only from libc or from the program's own pinned table, and an object inside someone else's program has neither.
Each make test-<os> links test/test_os_objlink.pas both ways with the
VM's cc (tools/vm/objlink.sh) and compares the output with the same
pair linked by gcc on Linux.
The /proc files the translators build
The BSDs and Minix have no Linux /proc (NetBSD's procfs and Minix's
/proc, when mounted, are different sets of files), so each translator
answers three more files from the system's own calls. The text is built in
compiler/builtin/proctext.inc, shared by all of them:
| File | Read by | FreeBSD | NetBSD | OpenBSD | DragonFly | Minix |
|---|---|---|---|---|---|---|
/proc/self/fdinfo/N (mnt_id:, ino:) |
the compiler's output guard | fstat |
fstat |
fstat |
fstat |
fstat |
/proc/mounts, /proc/self/mounts |
crtl's getmntent (df, mount lists) |
getfsstat |
getvfsstat |
getfsstat |
getfsstat |
getvfsstat |
/proc/stat (the cpu lines) |
palparallel's load sampler | kern.cp_time(s) |
kern.cp_time {1,51} |
kern.cp_time2 {1,71} |
kern.cp_time(s) |
kern.cp_time {1,51}, aggregate line only |
/proc/net/dev (the interface names) |
crtl's if_nameindex |
SIOCGIFCONF |
SIOCGIFCONF |
SIOCGIFCONF |
SIOCGIFCONF |
routing sysctl |
The interface ioctls behind crtl's getifaddrs and if_nametoindex
(SIOCGIFCONF, SIOCGIFFLAGS, SIOCGIFADDR, SIOCGIFNETMASK,
SIOCGIFBRDADDR, SIOCGIFINDEX, SIOCGIFNAME) are translated to the BSD
ones and their Linux layouts in compiler/builtin/ifnet.inc; an interface's
index comes from the AF_LINK entry of SIOCGIFCONF. OpenBSD's list has
none, so there it comes from the routing sysctl (NET_RT_IFLIST, what libc's
getifaddrs reads). Minix's network service answers SIOCGIFCONF with
ENOTTY, so the list itself is made from that sysctl (through the MIB
service), and the index from Minix's SIOCGIFINDEX. /proc/net/dev lists
the names with every counter 0.
mnt_id is the file system's device number, which is one per mounted file
system, as Linux's mount id is. A mount line has the source, the mount point,
the type, rw or ro with nosuid and noexec when set, and 0 0. The cpu
lines count user, nice, system, idle and interrupt time in the system's own
clock ticks (OpenBSD's spin time is added to system); Linux's other columns are
0. OpenBSD's own kern.cp_time is the average over the CPUs, so the first line
there is the sum of the per-CPU rows, as on Linux. Under --os=nt, winabi
answers all three: /proc/self/fdinfo/N (volume serial and file index),
/proc/mounts (the local drives, the current one on /) and the cpu line of
/proc/stat (GetSystemTimes, in 10 ms ticks).
Testing it
tools/vm/freebsd.sh runs a FreeBSD 15.1 VM under QEMU (KVM). The image is
several gigabytes. It goes in $PXX_VM_DIR/freebsd, which defaults to
/data/vm/freebsd; keep it off a small root filesystem.
tools/vm/freebsd.sh fetch # one time: download, verify, set up the ssh key
tools/vm/freebsd.sh start # boot headless; ssh on 127.0.0.1:2223
tools/vm/freebsd.sh run ./hello # copy in, run, relay stdout/stderr/exit code
tools/vm/freebsd.sh stop
run uses an 8 MB stack limit, as Linux does by default, and passes on any
PXX_TEST_* environment variables. PXX_VM_PTY=1 runs the program on a
terminal under script(1), for tests that need one.
make test-freebsd builds each listed test for both kernels, runs one copy
on the host and the other in the VM, and requires identical output. The
repository's hooks treat it as a full suite, so run it as
PXX_ALLOW_FULL_SUITE=1 make test-freebsd.
The compiler compiles itself on each of these systems. On 2026-11-15, the
--os=<os> build of compiler/compiler.pas, copied into each x86-64 VM with
the tracked compiler/ and lib/ trees, rebuilt itself there. Each result
was byte-identical to the Linux cross-build, in 54 s on FreeBSD, 57 s on
NetBSD, 84 s on OpenBSD and 74 s on DragonFly. The i386 --os=minix
build did the same in the Minix VM in 25 minutes. That needs compiler/
and lib/asmcore only, which fit the VM's small disk. Run natively, it makes
programs for its own system by default. There it also refuses an output
that is an input under another name, either sub/../t.pas or a hard link
(the check compares device and inode, through the translator's
/proc/self/fdinfo).
Things the VM does differently
These showed up as differences in the 2026-11-12 run and are not translator bugs:
- tests run as root, so permission checks (a mode-000 file, for example) pass where they fail for a normal user
- there is no
/etc/localtime /bin/shwords its error messages differentlyargv[0]is/tmp/pxxbin- free ports are assigned randomly
- stdout and stderr travel over ssh separately, so their order can differ
Changing the translator
Before you add to fbsdabi.pas:
- Take every number from the FreeBSD headers in the VM (
/usr/include/sys/), not from memory or a web page. Take the Linux side from the host's headers. - Check the Linux build stays byte-identical. The translator is only
compiled into
--os=freebsdprograms. A change to shared compiler code can still change Linux output, so compare a Linux build with the previous compiler. - Mind the order of checks. Linux and FreeBSD can give different errors
for the same bad call. For example,
socket()with a bad type isEINVALon Linux before the family is looked at. Copy Linux's order. - Some FreeBSD calls return the error as a value.
posix_fallocate, for example, does not set the carry flag. - The clone stub is compiler code.
compiler/thread_emit.inchas athr_newpath under--os=freebsd. Thread exit,CLONE_CHILD_CLEARTIDand the TID handshake are documented there. - The ELF header layout is shared. The FreeBSD note sits in the slot between
the program headers and the code, and the note must end where code starts.
--ldso-runnable(and--gibix) puts a 144-byte minimal dynamic block in the same slot, before the note. The layout is: headers, ldso block, FreeBSD note, code. Read the comment at the note inelfwriter.inc(both executable writers) before changing either.
The work log, with the reasons behind each choice, is in
devdocs/progress/working/feature-port-freebsd-native.md.
OpenBSD, NetBSD and Minix
mmap of a file goes to the kernel on every BSD, and msync's flags are
translated: Linux's MS_SYNC is 4, FreeBSD's and DragonFly's 0, OpenBSD's 2
(where MS_INVALIDATE is 4). Minix's memory service maps a file privately
or read-only, but answers ENXIO for a shared writable mapping. The
translator makes that mapping private and writes its bytes back to the file
on msync and munmap, never past the file's current end. Another process
sees the stores at those points, not before.
System V shared memory and semaphores work on Minix too, through its ipc
server (found by name through RS). Minix has no message queues, so msgget
and the rest say ENOSYS. The server also refuses SEM_UNDO, and so does
the translator: EINVAL, the answer Minix's own libc gives. It has no
semtimedop, so a timed wait retries semop every 10 ms.
OpenBSD, NetBSD, DragonFly and Minix have EXPERIMENTAL
--os=openbsd, --os=netbsd (since 2026-11-12; both also on arm64),
--os=dragonfly (since 2026-11-13, x86-64 only) and --os=minix
(since 2026-11-13, i386 only, phase 1).
On OpenBSD the default (since 2026-11-13, x86-64 and arm64) goes through
libc, OpenBSD's only supported system-call interface: see
through libc below. --rtl-raw
gives the static build described next instead.
--os=openbsd --rtl-raw (experimental): the static build
OpenBSD only lets a program make a system call from an address it has
declared in advance (pinsyscalls). The base system's static binaries do this
with a table in the executable, and that is what pxx writes. The design is the
FreeBSD one: the program is built against Linux, and
compiler/builtin/obsdabi.pas translates every system call at run time.
What is different from FreeBSD:
- The pin table. A
PT_OPENBSD_SYSCALLSprogram header (in the slotPT_GNU_STACKuses on Linux) points to a table after the data. The table pins every system call number, 1 to 330, to the singlesyscallinstruction inObRawA. A rawsyscallanywhere else in the program is killed with SIGABRT, and ktrace shows aPINSline. The kernel ignores the whole table if any entry is out of range, and it reads the table only when the program has noPT_INTERP. - No branch-target CFI. On a CPU with IBT (amd64) or BTI (arm64),
OpenBSD faults an indirect branch that does not land on an
endbr64orbtiinstruction, and pxx emits neither. The executable carries aPT_OPENBSD_NOBTCFIprogram header, the kernel's opt-out, as ports built without landing pads do. The VMs have neither feature (the build host has no IBT, and OpenBSD/arm64 does not boot on qemu's BTI-capable CPU models), so this part is read from the kernel source, not measured. - The ident note. The note is named "OpenBSD", type 1, description 0. It sits where the FreeBSD note goes. Without it OpenBSD answers ENOEXEC.
- TLS uses fs. OpenBSD userland cannot set the GS base;
__set_tcbsets FS. Under--os=openbsdthe compiler writes pxx's TLS accesses with the fs prefix instead of gs (TlsSegPrefixincompiler/defs.inc), and the translator turnsarch_prctl(ARCH_SET_GS)into__set_tcb. - No brk. OpenBSD has no brk call, so the translator emulates it with a
1 GiB
PROT_NONEreservation that it opens up withmprotect. /procfiles./proc/self/environis built fromsysctl kern.proc.args.<pid>.env. It is served through a pipe if it fits in 16 KiB, or through an unlinked file in/tmpif not (OpenBSD has no anonymous shared memory)./proc/self/exeisargv[0]made absolute. That is right when the program was started by a path, and it gives ENOENT when the program was found through aPATHsearch.- Release-bound. OpenBSD's system call numbers are not a stable ABI. The
numbers in
obsdabi.pasare 8.0's, measured from its/usr/include, and a binary is tied to that release.
Tested on 2026-11-12 on OpenBSD 8.0 amd64: hello world, and text-file
reading and writing, FindFirst, directory creation and removal, the
environment, the current directory, ParamStr(0) and the clock. The output
was identical to Linux.
Signals work the FreeBSD way: the translator installs its own entry point, which hands the program's handler a Linux-shaped siginfo and ucontext and copies the registers back. Three OpenBSD details matter:
- The signal mask is a single 32-bit word, passed by value.
- OpenBSD kills a process that makes a system call while its stack pointer
is outside a
MAP_STACKmapping. pxx's alternate signal stack is 32 KiB of BSS, butsigaltstackconverts the region it is given, so the program's buffer is registered unchanged. This was measured, not taken from the documentation. - OpenBSD 8.0 reports every SSE floating-point trap as
FPE_FLTINVand leaves the trap number 0. The translator works out the real cause from the saved MXCSR, the way Linux does.
test_signal_num, test_signal_sp_rewrite, test_stack_overflow_raise,
test_signal_bss_alias, test_signal_handler_callback_b336,
test_float_exception_mask and an --fpc-mem-errors nil write give the same
output on OpenBSD as on Linux (2026-11-12).
Threads work too (--threadsafe):
- Thread creation. The clone stub's OpenBSD leg calls
clonethrough the translator, which turns it into__tfork. That system call has its own pinned site, inObTforkA, because the child resumes after the instruction on its new stack and cannot return through the translator.ObTforkAsends the child to the stub's child entry. - Thread exit. Linux's kernel clears the tid word and wakes the joiner
(
CLONE_CHILD_CLEARTID); OpenBSD's does not. An exiting thread therefore takes a lock and moves to a shared exit stack. There it clears the word and wakes both futex queues, then calls__threxitwith the lock word as its argument, so the kernel releases the lock once the thread is gone. - Thread stacks must be
MAP_STACK. Under--os=openbsdthe compiler definesPXX_OS_OPENBSD, andpalthreadandscheduler(coroutines) then map their stacks with Linux'sMAP_STACKbit, which the translator passes on. A program that maps its own stack for a raw__pxxclonecall (astest_thread_clonedoes) must set that bit itself, or the thread is killed at its first system call. - futex. WAIT, WAKE and REQUEUE are translated, and the BITSET forms are accepted with a full mask.
test_a_threadvar_is_per_thread, test_cross_futex_through_the_pal,
lib_fpc_thread_surface (40 runs in a row), lib_classes_tthread,
test_signal_threads, test_signal_num_threads_race and the C tests
c_pthread_join_returns_value, c_errno_is_per_thread,
c_thread_local_is_per_thread, c_pthread_detach and
crtl_pthread_recursive_mutex give the same output on OpenBSD as on Linux.
Sockets, ppoll/pselect6 and epoll work too.
- Sockets use FreeBSD's translation (the structs are the same) with
OpenBSD's numbers.
AF_INET6is 24,SOCK_NONBLOCK/SOCK_CLOEXECare0x4000/0x8000, andMSG_NOSIGNALis0x400.SO_PEERCREDis reordered into Linux's pid, uid, gid. - epoll runs over kqueue, as on FreeBSD. OpenBSD's
struct keventis 32 bytes, not 64. - IPv6 is always v6-only on OpenBSD: a
[::]listener never accepts IPv4. Solib_net_v6only's two dual-stack checks fail there, and that is the system's behaviour, not a bug in the translation.
csocket_loopback_b88, c_crtl_sock_flags, c_crtl_cmsg_and_socket_levels,
c_crtl_inet6, lib_net, lib_net6, lib_net_timeout, lib_dns_tcp and
lib_asyncnet6 (coroutines over epoll) give the same output on OpenBSD as on
Linux.
termios and the tty ioctls use FreeBSD's tables, with OpenBSD's few
differences: there is no IUTF8, OCRNL/ONOCR/ONLRET are
0x10/0x40/0x80, and CRTSCTS is 0x10000. c_os_freebsd_termios gives
the same output as on Linux when run on a real terminal. Under OpenBSD's
script(1) with stdin redirected from /dev/null, the pty starts with every
flag at 0, because script copies the settings of a terminal it does not
have.
Some things OpenBSD lacks are emulated:
- timerfd is a kqueue holding one
EVFILT_TIMER. A kqueue fd is readable while it has a pending event, so epoll and poll see it fire; areadanswers the expiration count. pxx's scheduler parks every timed wait on a timerfd (CoSleep, the*Timeoutwaits, async DNS). Not emulated: a repeat interval different from the first expiry, and the valuestimerfd_gettimereports. /dev/fullis/dev/zerowith writes failing ENOSPC./proc/sys/kernel/hostnameis served fromkern.hostname.readlink("/proc/self/fd/N")works for a directory: the translator changes into it, reads the cwd and changes back. crtl'sfchdirrelies on this. For any other fd the answer is ENOENT.- Pseudo-terminals. Opening
/dev/ptmxopens/dev/ptmand asks it for a pair withPTMGET. The master is kept and the slave is closed.TIOCGPTNanswers the master's minor number N, and opening/dev/pts/Nopens OpenBSD's slave of that number,/dev/tty+pqrstuvwxyzPQRST[N div 62] +0-9a-zA-Z[N mod 62] (so/dev/pts/0is/dev/ttyp0). A stock/devonly has the 62pptys, so a 63rd pair fails with ENOENT.unlockptsucceeds, because OpenBSD ptys have no lock. fallocatemode 0 grows a short file withftruncate. OpenBSD cannot preallocate.- Scheduling. OpenBSD has no POSIX scheduling policies, so the
sched_*calls answer as Linux does for aSCHED_OTHERprocess.sched_getaffinityreports every online CPU.
make test-openbsd runs about 90 programs on the VM and compares their
output and exit code with the Linux build. It covers files, signals, threads,
sockets, async I/O, timers, termios (on a pty), ptys, and the C and NilPy runtimes.
Not done yet:
/proc/self/mapsand the rest of/proc, apart from the files above and the three in the /proc files.- eventfd.
- A thread stack mapped by the program itself for a raw
__pxxclonecall must haveMAP_STACK.
Anything that is not mapped answers ENOSYS.
To try it, run tools/vm/openbsd.sh start, then
tools/vm/openbsd.sh run <binary>. Set PXX_VM_PTY=1 to run the binary on a
real terminal. PXX_TEST_* environment variables are passed through.
OpenBSD/arm64. --os=openbsd --target=aarch64 (since 2026-11-13) uses the
arm64 front layer (compiler/builtin/lxa64front.inc) that FreeBSD and NetBSD
use, so obsdabi.pas keeps one translator for both CPUs. What is arm64's own:
- the system call: number in x8, arguments in x0..x7, carry set on error.
The kernel resumes 8 bytes past the
svc(libc'sdsb nsh; isbbarrier), so the same two instructions follow everysvcin the translator. The pin table names thesvcitself. - the signal frame is a
struct sigcontext(x0..x29, lr, sp, elr, spsr; no FP registers), converted to Linux's arm64 ucontext and back. - the kernel does not save
tpidr_el0when it switches threads; it reloads it from the value given to__set_tcb. The translator hands it the main thread's TLS block on the first system call;__tforkgives each thread its own. - text is mapped readable, as the program header asks, so the backend's
literal pools in the code segment work. BTI is switched off by the
PT_OPENBSD_NOBTCFIheader (see above).
make test-openbsd-arm64 runs about 85 programs on an arm64 VM (port 2231,
the stock miniroot80.img installed unattended into
/data/vm/openbsd-arm64, emulated with TCG; PXX_OBSD_ARCH=aarch64 selects it
in tools/vm/openbsd.sh) against the Linux aarch64 build under qemu-aarch64.
It passes.
--os=openbsd, the default: through libc
OpenBSD's supported interface is libc, not the system calls, so this is
what --os=openbsd builds (--rtl-libc says the same thing
explicitly; --rtl-raw asks for the static build above). The translator above still turns each Linux call into the
OpenBSD one, but its last step calls libc's function of the same name
(read, openat, kevent, ... 142 of them) instead of executing syscall.
The output is an ordinary dynamic executable:
-
PT_INTERPis/usr/libexec/ld.soandDT_NEEDEDislibc.so.104.0(OpenBSD 8.0). OpenBSD bumps libc's major number when its ABI changes and expects programs to be rebuilt; a binary for one release is not promised to run on the next. All imports are bound at load time (GLOB_DAT, no PLT). -
No pin table: ld.so pins libc's system calls, and the program makes none. (One raw
syscallbyte pair remains, the Linuxrt_sigreturnstub, which is never reached on OpenBSD; the static build carries it too.) -
A
PT_PHDRheader: OpenBSD's kernel gives ld.so the program headers' address only through it. -
libc owns
fs(its thread block is atfs:0, errno atfs:32). pxx's TLS block is the executable'sPT_TLSinstead, so ld.so places it right belowfs, and every pxx TLS access isfs:[slot - span], where span is the block size rounded to 16 (TlsSegDispBiasindefs.inc). The main-thread install takes the block fromfs:0 - spanand makes no__set_tcbcall. -
errno comes from
__errno(), cleared before each call so a legitimate -1 (getpriority) is not taken for an error.sigprocmask,sigpendingandsigsuspendtake pointers in libc and values in the kernel; the translator converts them. -
Threads start the way libc's own
pthread_createstarts them:_dl_allocate_tibgives the new thread a libc thread block with its pxx block below it, and__tfork_threadstarts it at a trampoline that resumes the clone stub's child entry on the stage area. The child'sarch_prctl(ARCH_SET_GS)copies the staged block into the real one, so in this mode the stub fills the staged block's thread-locals first. Thread exit is the static build's (clear the tid word, wake the joiners, then__threxiton the shared exit stack), with its calls through libc. A thread cannot free its own libc thread block (it uses it until__threxit), so each exiting thread frees the previous exiter's, under the exit lock the kernel releases only once that thread is gone; at most one block is left over. ld.so's allocator is not thread-safe by itself (libc's pthread normally serialises it), so_dl_allocate_tiband_dl_free_tibrun under a lock of the translator's own.
arm64 (--target=aarch64) works the same way, with TLS variant I:
tpidr_el0 points at libc's two-word TCB and the executable's PT_TLS (the
pxx block) starts 16 bytes above it, on every thread, so every pxx
thread-local access is tpidr_el0 + 16 + slot (TLS_A64_LIBC_OFF). The main
thread makes that block a pxx block (self-pointer, init image) instead of
installing one. _dl_allocate_tib returns the TIB, which on arm64 sits below
the TCB: tf_tcb is the TIB plus 40 (TIB_TO_TCB, measured from
<tib.h>). A new thread's stub copies its staged block into the real one
inline. make test-openbsd-arm64-libc runs test-openbsd-arm64's lists
this way; it passes.
Why libc is the default: OpenBSD does not keep old system calls once libc
stops using them, so a static binary that calls the kernel directly can stop
working on a later release with no warning. A binary on libc fails at load
time instead, naming the missing libc.so, and a rebuild fixes it. The
static build stays for systems without ld.so or for comparison.
make test-openbsd and make test-openbsd-arm64 test the static build (--rtl-raw); make test-openbsd-libc runs test-openbsd's lists: 57 programs, the 15
thread programs, and the environment and -g legs. It passes.
--os=netbsd (experimental)
The FreeBSD design again: the program is built against Linux, and
compiler/builtin/nbsdabi.pas translates every system call to NetBSD 11.1.
The executable carries a NetBSD ident note. NetBSD is the closest of the BSDs
to Linux: it has native timerfd, eventfd, futex, epoll, memfd_create
and dual-stack IPv6, and its wait statuses, UTIME_*, getrandom flags and
scheduling policies have Linux's values.
What the translator still has to do:
- System call numbers. Many calls have versioned names (
__fstat50,__getdents30,__socket30,__sigaction_sigtramp). The plain numbers are the old ABIs. PADarguments.mmap,lseek,pread,pwrite,preadv,pwritev,truncate,ftruncate,posix_fallocateandmknodattake a padding argument before their 64-bit one.mmap's offset is then the seventh argument, which NetBSD reads from the user stack, so the kernel entry always pushes a seventh word.epoll_eventis 16 bytes in NetBSD's kernel (data at offset 8), not Linux's packed 12, soepoll_ctlandepoll_waitrepack it. NetBSD ships no<sys/epoll.h>; the layout andEPOLL_CLOEXEC(=O_CLOEXEC) were measured with the raw system calls.- Directory entries are up to 8 bytes shorter than Linux's, so
getdents64reads into a buffer two thirds the caller's size and widens. - TLS lives in fs, set with
_lwp_setprivate, becausesetcontext, which ends every signal handler, restores the fs base from the saved context. With no libc in the program, fs is free. - Signals are installed with
__sigaction_sigtramp. The translator's entry builds the Linuxsiginfo/ucontext, calls the program's handler and resumes withsetcontextitself. - errno 1..84 are FreeBSD's; 85..98 differ. Linux's SIGPWR is NetBSD's 32, and the real-time signals are shifted by one (NetBSD has 33..63).
- Threads. The clone stub calls
clonethrough the translator, which starts an LWP with_lwp_createfrom a ucontext made bygetcontext:ripis the stub's child entry,rspthe new stack, and_UC_TLSBASEsets the thread's fs base. LWPs are created detached; the tid word is cleared and its waiters woken by the exiting thread itself, with its signals blocked and without touching its stack. /proc/self/environ,/proc/self/exeand/proc/sys/kernel/hostnamecome from sysctl, served through a memfd;readlink("/proc/self/fd/N")usesfcntl(F_GETPATH).- ptys are NetBSD's own
/dev/ptmxand/dev/pts/N;TIOCGPTNreads the number fromTIOCPTSNAME, andunlockptdoesTIOCGRANTPT.
make test-netbsd runs about 90 programs on the VM and compares their output
and exit code with the Linux build: files, signals, threads, sockets (IPv4,
IPv6, dual-stack), async I/O, timers, termios on a pty, ptys, -g, and the C
and NilPy runtimes. It passes.
Not done yet: /proc apart from the files above and the three in
the /proc files.
The VM is the stock NetBSD-11.1-amd64-live.img, booted under OVMF with
consdev=com0 in /EFI/NetBSD/boot.cfg on its EFI partition, and set up over
the serial console (sshd, dhcpcd, a root key). Run tools/vm/netbsd.sh start,
then tools/vm/netbsd.sh run <binary>; it listens on port 2227.
NetBSD/arm64. --os=netbsd --target=aarch64 works the same way and shares
FreeBSD/arm64's pieces: the compiler's svc thunk and the translator front
in compiler/builtin/lxa64front.inc, which maps Linux's arm64 system calls
onto the x86-64 ones nbsdabi.pas translates. What is NetBSD's own on arm64:
- the system call number is encoded in
svc's immediate; a translator computes numbers, so it usessvc 0(SYS_syscall) with the number inx17; - the main thread's thread pointer is written by the program's entry code,
which the kernel does not see, so the first system call after it reports it
with
_lwp_setprivate(otherwise thesetcontextthat ends a signal handler would reset it to 0); - threads are LWPs created from a
ucontextwhose_REG_TPIDRis the new thread's TLS block; - the signal frame's registers (
x0..x30,sp,pc,spsr) are in Linux's order.
make test-netbsd-arm64 runs about 75 programs on an arm64 VM (port 2229,
the stock evbarm-aarch64 arm64.img under /data/vm/netbsd-arm64, emulated
with TCG) against the Linux aarch64 build under qemu-aarch64. It passes.
--rtl-libc alone is not enough on any of them: it passes Linux system call
numbers to libc's syscall(), which OpenBSD's libc does not even export.
NetBSD and Minix have stable system call numbers.
The measurements and the plan are in
devdocs/progress/working/feature-port-libc-family-openbsd-netbsd-minix.md.
--os=minix (experimental, phase 1)
./pxx --target=i386 --os=minix hello.pas hello
Minix 3.4 has no system calls in the Unix sense. A process sends a 64-byte
message to a server (PM for processes and time, VFS for files, VM for
memory, MIB for sysctl) with int $0x21 and gets the answer back in the
same buffer. compiler/builtin/minixabi.pas turns each Linux i386 call into
such a message: under --os=minix every int $0x80 pxx emits is a call to
the shared thunk, which passes the Linux number and arguments to the
translator. Each message layout comes from Minix's own libc wrapper for the
call. The executable needs no note.
What the translator does that Minix does not:
- getcwd walks up through
.., as Minix's libc does (VFS has no getcwd). O_DIRECTORYandO_NOFOLLOWare accepted by Minix and ignored, so the translator checks the file itself; a 0-bytereadon a directory answersEISDIRas Linux does.execvebuilds the new process's stack in the caller, as Minix's libc does, and PM copies it in.- No
*atcalls: a directory fd other thanAT_FDCWDworks only with an absolute path. /proc:/proc/self/environcomes from the initial stack (the entry code passes it to the translator),/proc/self/exefrom the aux vector,/proc/self/fd/Nonly for directories,/proc/sys/kernel/hostnamefrom sysctl.- TLS: Minix cannot point
fsat a block, so the TLS base is the main block's address. That works because Minix has no kernel threads:--threadsafeis refused. - Signals: Minix calls a handler as
handler(signo, code, sigcontext*)and has noSA_SIGINFO. The translator installs its own entry, which builds Linux's siginfo and ucontext, calls the program's handler, copies any register changes back and returns throughPM_SIGRETURN. Handlers run on the interrupted stack, because Minix has nosigaltstack, so a stack overflow cannot be caught. A fault's address andsi_codeare not reported.sigtimedwaitpolls the pending set every 10 ms, andnanosleepisselectwith a timeout, as in Minix's libc.
Phase 1 covers files, directories, memory, time, fork/exec/wait, pipes,
signals, select/poll, vectored and positioned I/O, statfs, utimensat,
priorities, sockets, epoll/timerfd/eventfd and the environment. select,
poll and ppoll are Minix's select (descriptors up to 254). readv,
writev and pread/pwrite loop and seek as Minix's libc does.
-
Sockets are Minix's own VFS socket calls (TCP, UDP, IPv6, Unix domain), with NetBSD's constants: the same conversions as
--os=netbsd. -
epoll, timerfd and eventfd do not exist on Minix. Each is a real descriptor (on
/dev/null) whose state the translator keeps;epoll_waitselects on the real descriptors and wakes for the nearest timer. Level-triggered only (EPOLLETacts as level), and the state is per process. -
ptys are Minix's own
/dev/ptmxand/dev/pts/N.unlockptis Minix'sTIOCGRANTPT, andTIOCGPTNreads the number fromTIOCPTSNAME. -
alarmandsetitimerare PM's interval timers.prctlkeeps the task name forPR_GET_NAME. Thesched_*calls report one CPU and SCHED_OTHER.mlockandunshare(0)succeed and do nothing.
Minix itself has no /dev/full, no job control (SIGSTOP ends the process)
and no threads.
make test-minix runs 55 programs (Pascal, C and Nil Python) on a Minix VM and
compares their output and exit code with the Linux i386 build. It passes.
The VM is the stock minix_R3.4.0rc6 ISO installed under /data/vm/minix
with openssh from pkgin. It runs under TCG, because the install stalled
under KVM. Run tools/vm/minix.sh start, then tools/vm/minix.sh run <binary>;
it listens on port 2232.
--os=dragonfly (experimental)
compiler/builtin/dfbsdabi.pas translates every system call to DragonFly BSD
6.4. The executable carries a DragonFly ident note. DragonFly forked from
FreeBSD 4, so the system call numbers, errno 1..92, the signal frame and
stat are close to FreeBSD's but not the same. It is x86-64 only: DragonFly
has no other CPU. What is DragonFly's own:
- Padded calls.
lseek,truncate,ftruncateandmmapstill take the old pad argument before the 64-bit offset;pread/pwriteand theirvforms go toextpread/extpwrite. mmap's offset is the seventh argument and is read from the user stack. AT_FDCWDis -328243, not -100.- Threads are
lwp_create2. The kernel gives each new LWP a small id that is reused at once, so pxx hands outlwpid + generation << 16as the tid and keeps the generation in the thread's TLS. Thread exit clears the child-tid word, wakes it withumtx_wakeupand leaves withextexit, on a shared exit stack under a spin lock. Futexes areumtx_sleepandumtx_wakeup;umtx_sleephas no requeue, soFUTEX_REQUEUEwakes all. - TLS is
set_tls_areaon%gs, as on Linux. pipe2returns both descriptors in registers.getdents64isgetdirentries;/proc/self/exeand/proc/self/environcome from sysctl (kern.proc.pathname,kern.ps_strings).- Signals. The ucontext is FreeBSD's shape (mcontext at 64) with
DragonFly's register order. A SIGFPE's
si_codeis worked out again from MXCSR when the kernel leaves it at 0. dup3isfcntl(F_DUP2FD[_CLOEXEC]);fallocateisftruncate;prctl(PR_SET_NAME)islwp_setname.- IPv6 sockets are always v6-only:
IPV6_V6ONLYcannot be cleared, so a::listener does not take IPv4 connections. - ptys are
/dev/ptmxand/dev/pts/N; the pty number comes fromFIODNAME.
make test-dragonfly runs about 90 programs on the VM (the lists of
make test-netbsd minus lib_net_v6only) and compares their output and
exit code with the Linux build. It passes. The VM is the stock
dfly-x86_64-6.4.2_REL.img under /data/vm/dragonfly, booted by SeaBIOS
with a serial console and set up from its console (sshd, DHCP, a root key,
root's shell /bin/sh, noatime off). Run tools/vm/dragonfly.sh start,
then tools/vm/dragonfly.sh run <binary>; it listens on port 2230.