Add FFI_TYPE_VECTOR: vector (SIMD) type support with libffi-computed layout#1000
Open
edusperoni wants to merge 5 commits into
Open
Add FFI_TYPE_VECTOR: vector (SIMD) type support with libffi-computed layout#1000edusperoni wants to merge 5 commits into
edusperoni wants to merge 5 commits into
Conversation
Introduce a portable API for marshalling vector (SIMD) types -- the values produced by GCC's __attribute__((vector_size)) and Clang's ext_vector_type. This answers the stalled PR libffi#414 and the maintainer's 2018 design questions (https://sourceware.org/legacy-ml/libffi-discuss/2018/msg00020.html): rather than requiring callers to hand-compute a vector's size and alignment (and gating the feature behind configure), libffi now derives the layout itself and the type code is defined unconditionally. A vector is described exactly like a struct: type == FFI_TYPE_VECTOR and a NULL-terminated elements[] array, except every element must point to the SAME fundamental scalar (float, double, or a fixed-width integer UINT8..SINT64) and the count is the number of lanes. The caller leaves size and alignment at zero; ffi_prep_cif computes: size = lane_size * count, rounded up to the next power of two (matches Clang ext_vector_type storage: 3 x float -> 16, 3 x double -> 32; GCC vector_size already requires pow2 totals so it is identical there); alignment = min(size, 16). Validation (identical scalar lanes, count >= 1, scalar-only) yields FFI_BAD_TYPEDEF otherwise. - include/ffi.h.in: FFI_TYPE_VECTOR = 18 (after SINT128 = 17), FFI_TYPE_LAST bumped. Defined unconditionally, no configure gating. - src/prep_cif.c: initialize_vector() computes the layout in initialize_aggregate; ffi_type_contains_vector() rejects vectors (including nested in structs, argument or return) with FFI_BAD_TYPEDEF on any port that does not define FFI_TARGET_HAS_VECTOR_TYPE -- no aborts. Vector returns reserve the hidden return-pointer slot like structs. - src/raw_api.c, src/java_raw_api.c: plumb FFI_TYPE_VECTOR alongside FFI_TYPE_STRUCT, mirroring how FFI_TYPE_COMPLEX is handled. - src/debug.c: ffi_type_test requires elements != NULL for vectors. - src/pa/ffitarget.h: bump the FFI_PA_TYPE_LAST tripwire; PA gates vectors out in prep_cif so its jump tables are never reached. - doc/libffi.texi: new "Vector Types" node documenting the API, the computed-layout rule, the psABI framing, and the per-port support table. No port defines FFI_TARGET_HAS_VECTOR_TYPE yet, so this commit rejects every vector signature; the per-architecture ports follow. References: libffi#414, libffi#773.
Define FFI_TARGET_HAS_VECTOR_TYPE for AArch64 and teach is_vfp_type to
classify FFI_TYPE_VECTOR, so ffi_call and closures pass and return
vectors the way AAPCS64 (and current GCC/Clang) do:
- 8- and 16-byte vectors travel in a single V/Q register (a Short
Vector), for float, double and integer lane types alike;
- homogeneous vector aggregates -- a struct of up to four identical
8- or 16-byte vectors -- travel in that many consecutive V/Q
registers (an HVA), e.g. struct{float32x4 a,b} in {q0,q1};
- a bare vector wider than 16 bytes (e.g. a 32-byte double4) has no
short-vector register class, so is_vfp_type returns 0 and the
existing composite path passes it by invisible reference and returns
it in memory -- exactly what a natively compiled callee expects.
is_simd() reports the width of one Neon register slot (a bare vector's
whole size, or one lane vector of an HVA); is_vfp_type() encodes
num_registers slots of that width onto the existing AARCH64_RET_{D,Q}*
codes via intlog2. is_hfa0/is_hfa1 recurse through FFI_TYPE_VECTOR so
HVA homogeneity is checked, and the three fundamental-type dispatch
switches (machdep return, ffi_call_int, ffi_closure_SYSV_inner) route
FFI_TYPE_VECTOR through is_vfp_type alongside FFI_TYPE_STRUCT.
Ported from the battle-tested NativeScript aarch64 vector marshaller,
adapted to the FFI_TYPE_VECTOR API and extended so that integer-lane
vectors (e.g. int32x4) are classified into V registers too -- the
original only handled floating-point lanes.
References: libffi#414, libffi#773 (aarch64 vec4 return).
Define FFI_TARGET_HAS_VECTOR_TYPE for the SysV x86-64 backend (ffi64.c;
32-bit x86 and the Windows ffiw64.c backend are excluded) and integrate
FFI_TYPE_VECTOR into the existing psABI classifier without restructuring
it:
- classify_argument gains a FFI_TYPE_VECTOR case: an 8-byte vector is
one SSE eightbyte (X86_64_SSE_CLASS); a 16-byte vector is one %xmm
register (X86_64_SSE_CLASS + X86_64_SSEUP_CLASS). The existing
INTEGERSI/SSESF/SSEDF/UINT128 handling is untouched, and the SSE+SSEUP
argument marshalling already merges both eightbytes into one %xmm.
- ffi_prep_cif_machdep classifies vector returns symmetrically: 8 bytes
in %xmm0 (UNIX64_RET_XMM64), 16 bytes in %xmm0 (UNIX64_RET_XMM128).
- Vectors wider than 16 bytes return FFI_BAD_TYPEDEF from
ffi_prep_cif_machdep, for both returns and arguments. Correct
%ymm/%zmm passing needs unix64.S register-save changes and is left as
a v1 limitation rather than silently passing them in memory.
Closures need no separate change: the closure paths reuse
classify_argument for arguments and cif->flags for the return.
References: libffi#414.
Model a new testsuite/libffi.vector/ directory on testsuite/libffi.complex:
vector.exp reuses the same dg/run-many-tests driver and skips every test as
"unsupported" on ports whose headers do not define
FFI_TARGET_HAS_VECTOR_TYPE (libffi_feature_test), so unsupported targets
still compile the gating cleanly.
Vector types are built with the portable __attribute__((vector_size)) via a
small make_vector_type() helper (vector.h); each test cross-checks the value
returned through ffi against a direct native call. Coverage:
- vector_float32x4 / vector_float32x2 / vector_double2 / vector_int32x4:
pass and return 8- and 16-byte float, double and integer vectors
(float32x4 is the vec4 shape of libffi#773);
- vector_args_spill: ten vectors interleaved with int/double scalars,
exhausting the vector argument registers and spilling to the stack;
- vector_vec3: Clang-only ext_vector_type(3), verifying the 12->16 byte
power-of-two padding matches a natively compiled callee (a no-op on
other compilers);
- vector_double4: on AArch64 a 32-byte vector round-trips (by reference /
in memory); elsewhere ffi_prep_cif must return FFI_BAD_TYPEDEF, checked
for both return and argument;
- vector_hva: a struct of two identical vectors (HVA) passes and returns
on both AArch64 (Q-register pair) and x86-64 (SSE struct classification);
- cls_vector: a closure receiving vector arguments and returning a vector;
- vector_validate: heterogeneous lanes, an empty vector, and a non-scalar
lane are each rejected with FFI_BAD_TYPEDEF, and a well-formed vector is
accepted with the computed power-of-two size and min(size,16) alignment.
The files are added to testsuite/Makefile.am EXTRA_DIST, matching how
libffi.complex is distributed.
References: libffi#414, libffi#773.
Two fixes for the libffi.vector suite: - vector_double4.c: the non-aarch64 branch built its own void_args array and never read the already-populated args, tripping gcc's -Wunused-but-set-variable (an excess-errors FAIL on Linux x86-64 with gcc; clang does not emit this warning). Use args for the negative argument-passing check instead. - vector.exp: the suite only probed FFI_TARGET_HAS_VECTOR_TYPE, but the tests are written with the GCC/Clang vector extension. On Windows ARM64 the aarch64 port enables the feature while MSVC cannot compile __attribute__ ((vector_size)), so every test failed to build. Add a compile probe and mark the suite unsupported when the compiler lacks the syntax.
Author
|
Pushed 19dbdb5 with two testsuite fixes:
The Windows 64-bit MSVC failures look unrelated to this PR: they're all |
Author
|
@atgreen can you approve the workflows again so we're green? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a portable API for marshalling vector (SIMD) types — the values produced by GCC's
__attribute__((vector_size(N)))and Clang'sext_vector_type— with ports for aarch64 and x86-64.It supersedes #414 (open since 2018) and is designed as a direct answer to the design questions Anthony raised on the mailing list at the time (https://sourceware.org/legacy-ml/libffi-discuss/2018/msg00020.html), which is where that PR stalled. It also addresses #773 (aarch64
vec4return values), whose reporter had no way to describefloat32x4_tto libffi.The design questions, answered
The 2018 objection to #414 was that it required callers to fill in a vector's size and alignment by hand, breaking libffi's promise to compute type layout itself, and that GCC and Clang disagree about vector alignment. This PR takes the third option floated in that thread — "only require the definition of one element type":
type = FFI_TYPE_VECTORand a NULL-terminatedelements[]array — except every element must point to the same fundamental scalar (float, double, or a fixed-width integer), and the count is the number of lanes. No new struct fields, no ABI change toffi_type.size/alignmentat zero;ffi_prep_cifcomputes them:size = lane_size × lanes, rounded up to the next power of two (this is Clang'sext_vector_typestorage rule — e.g. 3 × float → 16; GCC'svector_sizealready requires power-of-two totals, so the rule is identical there);alignment = min(size, 16).Invalid descriptions (heterogeneous lanes, aggregate lanes, long double, zero lanes) return
FFI_BAD_TYPEDEF.Structure — the FFI_TYPE_COMPLEX precedent
The feature follows the same shape as complex-type support:
FFI_TYPE_VECTOR = 18appended afterFFI_TYPE_SINT128,FFI_TYPE_LASTbumped; defined unconditionally (no configure gating).FFI_TARGET_HAS_VECTOR_TYPEinffitarget.h. On any port that doesn't,ffi_prep_cifreturnsFFI_BAD_TYPEDEFfor vector signatures (including vectors nested in structs) — a clean error, never an abort.raw_api.c/java_raw_api.c/debug.calongsideFFI_TYPE_STRUCT, mirroring complex.Ports
aarch64 (AAPCS64): 8/16-byte short vectors in single V/Q registers (float, double, and integer lanes); homogeneous vector aggregates (structs of 2–4 identical short vectors) in consecutive Q registers; bare vectors wider than 16 bytes by reference, matching compiler codegen (verified against Clang). Closures handled symmetrically.
x86-64 (SysV psABI): additive
FFI_TYPE_VECTORcase inclassify_argument— 8-byte vectors classify as one SSE eightbyte, 16-byte as SSE+SSEUP (one%xmm); returns in%xmm0. Vectors wider than 16 bytes are rejected withFFI_BAD_TYPEDEFrather than passed subtly wrong: correct%ymm/%zmmpassing needs register-save changes inunix64.Sand is left as a documented follow-up. Existing classification (including the recent UINT128/SINT128 support) is untouched.Tests and docs
testsuite/libffi.vector/modeled onlibffi.complex/(same driver and target-skip mechanism): pass/return of float32x2/x4 (the On Android platform aarch64 ABI does not return vec4 correctly #773 shape), double2, int32x4; mixed scalar/vector calls with stack spill; the Clang 3-lane padding case; 32-byte double4 on aarch64; HVA structs; closures both directions; and negative tests for the validation and the x86-64 width limit.libffi.texidocumenting the API, the layout rule, the psABI framing, and the per-port support matrix.Provenance
The aarch64 marshalling logic derives from the NativeScript iOS runtime's production libffi fork, which has shipped vector marshalling for Objective-C
simd/SceneKit types since 2019. The runtime has now been ported to this exact API (FFI_TYPE_VECTORwith lane-countelements[]and computed layout) and passes its full test suite against this branch on arm64 device and simulator — so this PR is battle-tested marshalling reworked to the computed-layout design asked for in 2018, and merging it lets that fork be retired in favor of upstream.Happy to adjust the API surface or split/squash the series however you prefer.
cc @ronaldoussoren (PyObjC interest on #414), refs #414, closes #773