← board

T reports test_nilpy_str_isnumeric_istitle RED at full tier; not reproducible locally

devdocs/progress/tstate/plexus.json at 2026-08-09T08:30:09Z:

history[-1]: { sha: 0d6de0cbe, tier: "full",
               new_red: ["test-nilpy#src:test/test_nilpy_str_isnumeric_istitle.npy"] }
open_regressions[0]: { job: same, good: 4939f47ab, bad: 0d6de0cbe, range: 14 commits }

The test was added in 38b94a77d (inside that range) together with the pystr_isnumeric / pystr_istitle implementations.

What was checked locally, at the same sha, clean tree

So the red is not reproducible on this box.

The one thing that looks explanatory, and does not hold up

pin_shadow at the same timestamp reports unexpected: [ selfhost-fixedpoint#src:compiler/compiler.pas, test-nilpy#src:...isnumeric... ] with would_pin: false. A new-feature test failing under the PINNED compiler is expected between landing and a re-pin, and pin_shadow's own comments describe the stale selfhost-fixedpoint key as a known orphan. That would explain the shadow entry — but the history entry records it as a new_red in the full tier, which is the real run, not the shadow. So the shadow explanation covers only one of the two appearances.

Why this is filed under T rather than N

Per CLAUDE.md, T owns the tool and never the bug — and a finding is normally filed into the owning lane, which here would be N. It is filed under T instead because the finding itself is a disagreement between T's verdict and the same command run locally at the same sha, so the first question is whether the job's attribution or its environment is right, not what the NilPy code does. If T reproduces it with the failing output attached, re-file into N immediately and this ticket closes.

What would settle it: the actual diff output from T's run of that recipe. The tstate record carries the job name and shas but not the failing output, so there is nothing to diagnose from — which may itself be worth improving.

2026-08-09 (later) — T BISECTED it to one commit, and it is not the isnumeric one

twatch --status now reports bad=e3b8023ef736 (1 in range). That is the relative-imports commit, not the commit that added the isnumeric test.

That reframes it. In the Makefile the isnumeric recipe sits IMMEDIATELY BEFORE the relative-import recipe, so a failure in the latter attributed to the nearest preceding test source would name exactly this job. Both recipes pass locally in the same full run (log lines 284-286 and 290-292).

The structural difference is now removed. The relative-import recipe was the only one in the whole file shaped cd test && $(CURDIR)/$(COMPILER) .... It turns out the cd was never needed — pxx resolves the sibling helper modules relative to the SOURCE file, so the test compiles and passes from the repo root like every other one. The recipe is now an ordinary one-liner.

If the watcher's per-source job extraction was tripping on that cd (running the diff line without its compile line, say), this fixes the red. If the red persists at the next full run, the cd was not the cause and the ticket stands with its original evidence.

Worth keeping either way: a recipe shaped unlike its 300 neighbours is a bad thing to introduce, independent of whether it caused this.

2026-08-13 — RESOLVED by the condition this ticket set for itself

The ticket's own test was: "If the red persists at the next full run, the cd was not the cause and the ticket stands with its original evidence." It did not persist. tstate/plexus.json:

2026-08-13T06:04:44Z  8e5ed5e8b  full     new_red=[]  red=0
2026-08-13T07:17:27Z  4636171e0  full     new_red=[]  red=0
jobs["test-nilpy#src:test/test_nilpy_str_isnumeric_istitle.npy"] = "pass"

Two independent full tiers, four days after the recipe was normalised, with the job green in both and no open regression naming it. So the diagnosis holds: the red belonged to the neighbouring relative-imports recipe and was attributed to the nearest preceding test source, and removing the one recipe shaped unlike its 300 neighbours removed it.

Closed as done, not rejected: the finding was real and the fix was real. The part worth keeping is the ticket's last open observation — tstate records a job name and shas but not the failing output, so a T-only red has nothing to diagnose from and costs a local re-run to disprove. That is the general defect this instance exposed, and it outlives this ticket; it belongs with [[bug-t-tstate-launders-skip-into-pass]], which is the same class of "the published state does not say enough to be trusted".

Log