web: Add experimental wasm64 (Memory64) build target#23981
Conversation
There was a problem hiding this comment.
I don't think this is something we'll accept. The crate ecosystem should catch up instead.
There was a problem hiding this comment.
The branch for my use case (a Flex Enterprise application with a lot of XML) is invaluable because it allows me to overcome the 4GB memory limit.
But obviously this PR is only to stimulate discussion about enabling the WASM64 target, which has now been officially announced. The Ruffle codebase is practically ready for the 64-bit build. The only issues concern external crates, which we hope will be fixed with the small adjustments needed to make them 64-bit compliant.
9b4592a to
6c005e9
Compare
Build the web player for wasm64-unknown-unknown to lift the 4 GiB linear-memory ceiling of wasm32, which OOMs on XML-heavy content. Ruffle's own code is already pointer-width-agnostic; the blockers were dependencies whose cfg gates are written `target_arch = "wasm32"` instead of `target_family = "wasm"`, which excludes wasm64. This adds: - A `build:wasm64` npm script and a `buildWasm64` path in build_wasm.ts (target wasm64-unknown-unknown, -Z build-std, wasm-opt --enable-memory64). - tools/vendor_wasm64.ts: fetches and patches the affected crates (wgpu, wgpu-types, wgpu-core, wgpu-hal, wgpu-core-deps-wasm, glow, rfd, chrono, jpeg-decoder), widening their wasm32 gates and regenerating [patch.crates-io]. Vendored sources are gitignored and regenerated at build time; Cargo.lock is realigned automatically when a new entry lands in the table. Per-crate `skipPaths` allow narrowing the widening so wasm32-specific intrinsic modules (e.g. jpeg-decoder's SIMD path, which uses `std::arch::wasm32::*` and has no wasm64 equivalent) stay intact. - A getrandom custom backend (web/src/getrandom_custom.rs) backed by crypto.getRandomValues, since getrandom's wasm64 fix only exists in 0.4.x while rand 0.9 pins 0.3.x. All vendoring/patching is isolated and removable: as upstreams ship wasm64 support, drop entries from the VENDORED table until the shim can be deleted. Experimental. Renderer: webgpu + webgl (ANGLE).
|
I realize this PR is too specific to hope for acceptance. Similarly, other branches I'm developing, which implement heavy optimizations appropriate for enterprise Flex applications, will no longer be submitted as PRs but will remain available on my fork. |
|
I'm gonna try to slowly push this through, but it's gonna take a long time to bubble up into releases. So far, I filed: |
Motivation
The wasm32 build is capped at 4 GiB of linear memory, which is hit by
XML-heavy AS3 workloads (a real Flex 4.6 dataset OOMs at ~4.1 GB on
master). The Wasm Memory64 proposal is part of Wasm 3.0 and supported in
recent V8/SpiderMonkey, so a 64-bit build is now feasible.
This PR adds an opt-in wasm64 build next to the existing wasm32
extensions/MVP variants, with runtime selection unchanged.
What this PR is
An experimental, additive
BUILD_WASM64=true npm run build:wasm64that produces a Memory64 web build. Nothing about the existing wasm32
builds changes.
Validated end-to-end:
cargo buildonwasm64-unknown-unknown(release profile)wasm-bindgen(the 0.2.120 we pin already supports the wasm64 ABI)wasm-opt --enable-memory64Output
_bg.wasmis ~14 MB, with(memory $0 i64 …)confirmingMemory64.
The pattern
Ruffle's own code is pointer-width-agnostic — the only
target_archreferences in
core/src/already usetarget_pointer_width. Theblockers were all dependencies with cfg gates written
target_arch = "wasm32"where the intent is "any WebAssembly".On wasm64 those gates don't fire, web code paths get excluded, and
the build fails.
The fix is mechanical and identical for every affected crate:
What's included
web/packages/core/tools/build_wasm.ts: abuildWasm64branch thatinvokes
cargo build --target wasm64-unknown-unknown -Z build-stdwith
RUSTC_BOOTSTRAP=1(stable + bootstrap, no nightly toolchainrequired), then
wasm-bindgenandwasm-opt --enable-memory64.web/packages/core/tools/vendor_wasm64.ts: a small shim that fetchesthe affected crate sources (cargo cache offline-first, fallback to
crates.io), widens their wasm32 cfg gates to
target_family = "wasm",and regenerates a marker-fenced
[patch.crates-io]block. Idempotent,fails loudly if a crate stops needing the patch (so it can be removed
from the table). Per-crate
skipPathsallow narrowing the widening sowasm32-specific intrinsic modules (e.g. jpeg-decoder's SIMD path, which
uses
std::arch::wasm32::*and has no wasm64 equivalent) stay intact.The
vendor/directory is gitignored — sources are re-derived fromCargo.lockat build time;Cargo.lockis realigned automaticallywhen a new entry lands in the table. Currently patched:
wgpu,wgpu-types,wgpu-core,wgpu-hal,wgpu-core-deps-wasm,glow,rfd,chrono(wasmbindgate),jpeg-decoder(multi-threadedworker
not(wasm32)-gated).web/src/getrandom_custom.rs: a getrandom custom backend routedto
crypto.getRandomValues. Upstream's wasm64 fix(Add
wasm64-unknown-unknownsupport forwasm_jsbackend rust-random/getrandom#848) only exists in 0.4.x, whilerand 0.9pins 0.3.x — until a 0.3.x release ships it, we provide the symbol
ourselves.
How it's designed to be removed
Each entry in
VENDOREDcorresponds to one upstream cfg gate. When acrate ships native wasm64 support, drop its row from the table — the
[patch.crates-io]block shrinks automatically. When the table isempty, delete
vendor_wasm64.ts, the call site inbuild_wasm.ts,and the marker block in
Cargo.toml. The shim is self-policing: if acrate has 0 matches for the wasm32 gate, the script fails with a clear
message ("review and remove its VENDORED entry").
Caveats / known limitations
npm run buildis unchanged.wgpu-webgl. The WebGPU backend should workthe same way (same
wgpucfg gates are widened), but I haven't validated it.getrandomshim is sound (Web Crypto, same source as thewasm_jsbackend) but ideally lives upstream once 0.3.x ships Implement Avm1String #848.representative workloads.
Safari. The existing wasm32 builds remain the default fallback for everyone else.
Toolchain
Stable Rust (1.95+) with
RUSTC_BOOTSTRAP=1(the existing wasm webbuild already uses this pattern). Requires
rust-srcforbuild-std.Java in PATH for the playerglobal build (same as today).
Try it
rustup component add rust-src cd web npm run build:wasm64Open questions for reviewers
vendor_wasm64.tsapproach as an interim, vs.waiting for the upstreams to land their fixes?
to "supported"?
Checklist