← board

C test-corpus expansion: c-testsuite → zlib → tcc (+ csmith fuzz harness)

The plan, ranked by variation-per-effort

1. c-testsuite (FIRST — cheapest, broadest)

2. zlib (quick win, new workload class) — STARTED 2026-07-06

See [[feature-c-corpus-zlib]]: vendored (v1.3.1), test/zlib/runner.c + make test-zlib, gcc oracle green. Two compiler blockers filed ([[bug-c-typedef-name-as-uninitialized-local]] + a zlib.h gzgetc macro parse bug). libc-gap collector: [[feature-crtl-implement-libc-assumptions]].

3. tcc — Tiny C Compiler (the milestone)

4. csmith differential fuzzing (parallel, ongoing harness)

5. Duktape (later, if appetite)

COPY-PASTE KICKOFF PROMPT (fresh session)

You are Track A/C (compiler + C frontend), master. Task: land step 1 of the C corpus expansion — the c-testsuite conformance battery — per devdocs/progress/backlog/feature-c-corpus-expansion.md (read it first; order and rationale are settled with the user, do not re-litigate).

Verified facts (2026-07-06, do NOT re-derive): the suite is pure data — exactly 220 tests tests/single-exec/00001.c..00220.c, contract = main entry + NNN.c.expected matches stdout+stderr, plus NNN.c.tags metadata (C-standard level, portability, arch). The repo's own runner infra (sh/Python3/TAP/TMSU) is their CI only — bypass it; their per-compiler runners are 10-line CC=x wrappers + a .skip script, nothing more.

  1. Vendor https://github.com/c-testsuite/c-testsuite via the tools/install_lib_candidates.sh pattern (gitignored vendor source like sqlite/lua, PROVENANCE.md with upstream commit hash).
  2. Write OUR OWN runner (tools/ script + make test-c-conformance): for each test, compile ./compiler/pascal26 -Ilib/crtl/include -Ilib/crtl/src NNN.c, run with timeout, compare stdout+stderr against NNN.c.expected (byte-exact) and exit code 0. TAP output not needed; summary line pass/fail/skip.
  3. Triage every failure: (a) cfront/codegen bug — reduce to minimal repro (compare vs gcc), fix or file per lane rules (shared internals = Track A ticket; as A/C combined you may self-resolve); (b) legit unsupported feature — add to an EXPLICIT skip-list file (mirror their <name>.skip idea) with one-line reason sourced from the test's .tags where possible. No silent skips. Bug fixes >> skips; skip only what is genuinely out of scope (e.g. features we have consciously deferred).
  4. Gate: runner green (skips documented), make test + self-host byte-identical, test-lua-cross 24/24, sqlite suite still byte-identical vs same-version gcc oracle (oracle needs extra TU const char sqlite3_version[]="3.46.0"; — see done/bug-c-create-trigger-huge-alloc-oom.md for why). If compiler bugs were fixed: make stabilize + make pin (watch pin.log — a stabilize flake makes pin silently re-bless the OLD binary; verify VERSION advanced).
  5. Commit runner + skip-list + any fixes with regression tests; wire into make test only if fast (<~30s), else standalone target noted in this ticket. Update this ticket (step 1 done-notes), regenerate BOARD.md, push.

Steps 2 (zlib) and 3 (tcc) are separate sessions — do not start them unless step 1 lands early and all gates are green.

Cross-target skip lists CLEARED 2026-07-08 (A+C session)

All three 32-bit per-target skip lists (pxx.skip.{i386,arm32,riscv32}) are now EMPTY — every c-testsuite test that passes on x86-64 also passes cross. Fixed this session:

Skip list burned down to 3 — 2026-07-09 (cfront-agent, A+B+C)

x86-64 conformance now 217 pass / 0 fail / 3 skip (was 213 at session start, 172 baseline). Cleared this session, each with a regression test + self-host byte-identical:

Only 3 skips remain, all deep multi-session features:

Step 1 essentially COMPLETE — 219/220, only 00216 left — 2026-07-09 (A+B+C agent)

Conformance is now 219 pass / 0 fail / 1 skip (v180). Cleared this session, each self-host byte-identical + gcc-verified + regression test wired into test-core:

Only 00216 remains — and it needs FILE-SCOPE compound literals in array initializers (struct Wrap g[] = {((struct Wrap){f}), f} desyncs the top-level parser before any other 00216 feature is even reached). That is the large, self-host-fragile keystone; full 5-path map in feature-c-compound-literals. Step 1 is done bar this one deferred feature. Steps 2 (zlib) / 3 (tcc) unchanged.

2026-07-13 — step 4 (csmith) BLOCKED on a one-line install; everything else on the ladder is done

Steps 1-3 are complete (c-testsuite 220/220, zlib byte-identical to the gcc oracle, tcc self-compiles and the pxx/gcc lineages converge). Duktape is parked on [[bug-c-duktape-double-formatting]]. That leaves step 4, the csmith differential fuzzer — the highest-leverage item left here, because it is a one-time harness that then finds miscompiles on its own, overnight, forever.

It cannot be started unattended: csmith is not installed and sudo requires a password.

One command unblocks it (user):

sudo apt install csmith creduce      # csmith 2.3.0 is in the archive; creduce minimises hits

Once that is in, the harness is small: generate with csmith --output t.c, compile the same file with gcc -O2 -I/usr/include/csmith and with pxx, run both, diff stdout; on a mismatch, creduce down to a minimal repro and file it. csmith programs are self-checking (they print a checksum of all globals), so the oracle is just "same checksum", and its output is UB-free by construction, so a difference is always a real miscompile in one of the two compilers.

Log