wasm32-wasi support
#1179
Merged
wasm32-wasi support
#1179
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dhess
force-pushed
the
dhess/wasm
branch
7 times, most recently
from
ca61078 to
205c61b
Compare
This Nix dev shell adds the `ghc-wasm-meta` tools to the shell via the https://gitlab.haskell.org/ghc/ghc-wasm-meta flake. With `wasm32-wasi-ghc-9_6`, we can now build `primer` and `primer-api` with the Wasm backend target. (Note that these 2 packages are sufficient to build and run Primer programs, but lack any persistent storage and/or web service.) To get a Nix shell with these tools installed, run: ``` nix develop .#wasm ``` Then run `wasm32-waasi-cabal` to build targets. Caveats: * This only works on `x86_64-linux`, unfortunately, as the required GHC bindists aren't yet available for macOS. * The generated code isn't yet tested, as bringing up the test suite on Wasm is proving to be non-trivial. * There's no haskell.nix support here, meaning no binary caching, and no support for our CI suite, among other things. These tools are available only for interactive builds at the moment. * Though in theory any `primer` or `primer-api` function is now callable via Wasm, there is still a lot of work to be done before that's actually possible. Signed-off-by: Drew Hess <[email protected]>
Specifically: * `pretty-show` requires `happy` at build time, which doesn't work on Wasm targets. * `semirings` wants to define instances for a few Posix types, which aren't available on Wasm (WASI) targets. * `pretty-simple` uses a custom setup, which breaks on Wasm. In each of these instances, we resort to a fork and a `source-repository-package`. (At least 2 of these currently have upstream unmerged PR fixes.) Signed-off-by: Drew Hess <[email protected]>
Note that not all of these targets build successfully yet. Signed-off-by: Drew Hess <[email protected]>
Notes: * We "source repo" `splitmix`, which needs a few fixes that upstream doesn't want to merge at the moment. * We can't run `tasty-discover` via `wasmtime` at build time, so we have to bake the discovered tests out. We do this via a `Makefile` step, and `.gitignore` the generated test file. * The GHC runtime in Wasm doesn't seem to work with `-threaded` or `-rtsopts`, so these are disabled for that target. * `tasty` needs to be built for Wasm without `-unix`, as that tries to install signal handlers that aren't present in `wasmtime`. Signed-off-by: Drew Hess <[email protected]>
Signed-off-by: Drew Hess <[email protected]>
Previously, we checked for GHCJS and disabled tests accordingly. I think we're unlikely to use GHCJS at this point, so we remove these checks. I've also updated a related comment on why we add orphan instances for some OpenAPI 3 instances, because I believe they're necessary for Wasm support, in addition to GHCJS. Signed-off-by: Drew Hess <[email protected]>
Signed-off-by: Drew Hess <[email protected]>
Signed-off-by: Drew Hess <[email protected]>
Signed-off-by: Drew Hess <[email protected]>
Signed-off-by: Drew Hess <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This series of commits makes the necessary & sufficient changes to build the
primerandprimer-apipackages using GHC 9.6's WebAssembly target, and to get their tests to pass using thewasmtimeruntime. We expect these are the only 2 libraries we'll need in order to run Primer in the student's browser, as everything else is either very database or HTTP API-specific.There are a few hacks needed to make this work, but nothing that's particularly objectionable. The main thing I would like to clean up are the 4 additional
source-repository-packageoverrides we need for upstream Wasm-related fixes, but presumably those will come over time.There are currently
31 other major limitations:haskell.nixornixpkgs.haskellPackagessupport thewasm32-wasicross target at the moment. Therefore, we need to install the https://gitlab.haskell.org/ghc/ghc-wasm-meta flake and create anix developshell with thewasm32-wasi-*toolchain. This toolchain is currently only available forx86_64-linux, and doesn't use the Nix store to build packages, so it's not ideal, but should be good enough until one of the various Haskell Nix integrations supportswasm32-wasi.There's no CI integration yet, mainly due to point 1 and the fact that our Haskell CI is based entirely on Nix.(Fixed, though somewhat hacky.)The recently-merged(See chore: Disable animations support #1186, which addressed this.)Animationprototype (Animations #1164) pulls in dependencies that break thewasm32-wasibuild for a number of reasons (zlibandfsnotify, at least). Unfortunately, we didn't discover this until after we'd merged Animations #1164, otherwise we might have sat on it for awhile. As we expect that theAnimationimplementation will get a fairly substantial refactor, mainly by moving the interpreter out of the backend and into the frontend, it's tempting to revert Animations #1164 in favor of merging WebAssembly support intomain, but for now, we'll keep it, and relegate the WebAssembly support to this branch.