← board

PXX portable userland (mini OS-personality) — one shell, any kernel

The idea (one line)

PXX supplies the userland the kernel deliberately leaves out. The "kernel" below is just an ABI — Linux (syscalls) or FreeRTOS (tasks/IPC). The PAL ([[feature-platform-abstraction-layer]], landed) is the seam that swaps them, so one shell/app source runs as Linux PID 1, on ESP32/FreeRTOS, or as a desktop process — swap the PAL backend, not the app. This is the ir-as-substrate thesis made visible: thin frontend + shared IR + one platform seam ⇒ the app is kernel-agnostic by construction, not by luck.

Backends (the same userland, three kernels)

Flagship app — the NilPy shell (busybox/applet model)

A Unix-ish shell written in NilPy (Nil-Python): line editor → parse → applet dispatch → a | b | c. No fork/exec (no MMU on classic ESP32): one binary, commands are built-in applet functions dispatched by name (echo cat ls grep wc head tail ps), optionally run as tasks. A VFS gives the filesystem illusion (borrow from the sqlite file-VFS + PAL groundwork).

Why NilPy: it doubles as a frontend forcing function — the shell drags NilPy from "proven toy" (classes, control flow, auto-typing, C-import binding all work) to a real language, one concrete feature at a time. And it demos the cross-target thesis at the concurrency level (preemptive tasks on ESP, threads on desktop).

Why not busybox (decided — stop re-litigating)

busybox recurs in brainstorms as "the shell." It's welded to the Linux kernel model: fork/exec (multi-call binary re-invokes itself per applet), termios, signals, /proc, the full syscall + VFS surface. Porting it to ESP32/FreeRTOS (no processes, no MMU, no fork/exec, one address space) = reimplementing half a kernel — significant hacks, wrong effort. So busybox is NOT the ESP path; the hand-written NilPy applet shell is. Two legitimate busybox uses stay open, both Linux-only and separate from this demo's portable goal:

Real dependencies (NilPy frontend gaps the shell forces — Track A)

Phasing

  1. Desktop-first, NilPy: shell loop + parse + applet dispatch + a|b via buffers, on x86-64. Surfaces the collection/string gaps → drive them.
  2. Applet set: echo cat ls grep wc head tail ps over a VFS abstraction.
  3. Cross to ESP: desktop PAL → IDF PAL; applets as FreeRTOS tasks; pipes as stream buffers.
  4. Polish: history, redirects, & background = spare task.

Gate (file-ownership Track B for the app; Track A for the NilPy deps)

App/demo builds with $(PXX_STABLE) (Track B rule — never rebuild the compiler); make lib-test / demos green; desktop shell runs; ESP variant boots under the QEMU/IDF harness. NilPy-frontend deps carry Track A's gate (self-host byte-identical) since they touch shared frontend/RTL.

[[feature-platform-abstraction-layer]] (the seam) · [[feature-kernel-matrix-bootroom]] (Linux backend) · [[feature-nilpy-collections-and-string-methods]] (blocker) · [[project_nil_python_arc]] · thesis devdocs/dev/ir-as-substrate.md.

Log

2026-08-09 (Track B): phase 1 LANDED — every listed blocker was already gone

The 2026-07-20 log said phase 1 was gated on a Track A re-pin past c6505149 and that there was "nothing to do here until make pin moves". The pin is v252 now, so the first thing was to re-run the stated check:

$ $(PXX_STABLE) examples/shell/shell0.npy && ./shell0
applets: echo wc upper rev help
hello portable userland
...

It prints the canned session correctly. Blocker cleared.

Then every OTHER gap the ticket lists was re-measured rather than assumed, and all of them are gone too:

listed gap state on the current pin
list + append (argv, job table) works
dict works
str.split works
str.join works
str.strip works
import sysutils tripping "array of const requires builtinheap" works
no stdin/sys surface import sys works

So [[feature-nilpy-collections-and-string-methods]] — recorded here as "the hard blocker; file/advance this first" — is satisfied in practice.

examples/shell/nilsh.npy — phase 1

Written the way this ticket always wanted it, now that the language allows it: argv as a LIST, parsing by split(), applets as functions with the (argv, stdin) -> stdout signature. Roughly half the size of shell0.npy, which had to hand-roll a character scanner for tokens.

Pipes are real, which was the phase-1 goal: a | b | c runs each stage and feeds its output to the next. The session exercises a three-stage pipe (echo alpha beta | cat | upper). That applet signature is deliberately the shape a FreeRTOS stream buffer slots into for phase 3 — blocking reads give backpressure without hand-coded yields.

Applets: echo cat wc head tail grep upper rev help — the set this ticket lists, minus ls/ps, which want the VFS and a task table respectively (phase 2/3).

It is plain Python, so CPython is the oracle, and make lib-test now asserts that the SAME source prints byte-identical output under python3 and under pxx. That is the cross-runtime half of the thesis, checked rather than claimed.

shell0.npy is kept beside it as the phase-0 record of what the language could not do yet.

Phase 2 (2026-08-09): the VFS, ls and cat <file>

A dict is the whole filesystem, and that is the portable shape rather than a shortcut: classic ESP32 has no filesystem at all, so "files" have to live somewhere the app owns. ls lists it, cat <name> reads it.

On a hosted target cat falls THROUGH to a real file when the name starts with / — same source, richer backend, which is the thesis in one branch. On ESP that branch simply never matches. Swapping the dict for the IDF VFS is the phase-3 change and touches nothing above it, because the applets only ever call vfs_list / vfs_read / vfs_exists.

Measured while building it: NilPy has open/read/write/close and with open(...), so the passthrough is real — verified reading /etc/hostname. os.listdir does NOT exist, which is why ls is VFS-only; that is the honest boundary rather than a missing feature, since ESP would not have it either.

The session now exercises ls, cat motd, cat notes | wc, cat notes | grep beta and a missing-file error, and the whole thing is still byte-identical to CPython under make lib-test.

Next: phase 3, the ESP cross — applets as FreeRTOS tasks, pipes as stream buffers, the VFS backed by IDF. That one needs a device to finish, like the other ESP items.