A host method's result is read past its width on 32-bit targets
Rows affected, all same after the fix:
test_nilpy_open_world_method_dispatch:cross True Trueon i386 (it always was), andmixed True Trueon arm32 once the softfloat pull changed what the float compare left in r1.test_nilpy_pyeval_host_arity_and_returnson arm32:rising(3, 2, 1)came back True.test_nilpy_len_of_a_variant_shim_objecton arm32:68719476739(0x10_0000_0003) for 3, and the truth test read that as True.
Mechanism
TPFn*, TPMI_*, TIFn*, TNoArgI, TVarArgI and TStrArgI are all
declared to return Int64, whatever the callee really returns. On x86-64 the
callee writes the full register. On 32-bit the second word is whatever the
callee last left in r1 or edx.
Fix
PyNarrowRet(v, rk) in pylib.pas narrows a result to its RetKind:
- a byte for Boolean and Char;
- a sign-extended word for Integer;
- on CPU32, the low word for pointer, class and NativeInt kinds.
It is applied at every Int64-thunk result: pyeval's register, mixed and
TIFn families, and pylib's PyUserObjNoArgMeth, vi and gi arms.
Test
Rows in test-i386 and test-arm32 for the three tests above. All three are
wrong with v446 on arm32 (open-world also on i386).