From b6b31370021099724c7230ac89a84e46155d1f07 Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Sat, 13 Jul 2024 20:15:41 +0100 Subject: [PATCH 1/6] create gitignore in scheme-libs/racket/unison/ --- scheme-libs/racket/unison/.gitignore | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 scheme-libs/racket/unison/.gitignore diff --git a/scheme-libs/racket/unison/.gitignore b/scheme-libs/racket/unison/.gitignore new file mode 100644 index 0000000000..64e9064d19 --- /dev/null +++ b/scheme-libs/racket/unison/.gitignore @@ -0,0 +1,6 @@ +compiled/ +boot-generated.ss +builtin-generated.ss +compound-wrappers.ss +data-info.ss +simple-wrappers.ss From 3cd2a76d5e6744599232bc972128f9376ac0817d Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Sat, 13 Jul 2024 20:54:55 +0100 Subject: [PATCH 2/6] update scheme-libs/racket/unison/Readme.md --- development.markdown | 4 ++ scheme-libs/racket/unison/Readme.md | 64 +++++++++++++++++-- .../transcripts-manual/gen-racket-libs.md | 7 +- 3 files changed, 65 insertions(+), 10 deletions(-) diff --git a/development.markdown b/development.markdown index 962a507c63..fa5d613b84 100644 --- a/development.markdown +++ b/development.markdown @@ -187,3 +187,7 @@ nix develop '.#cabal-unison-parser-typechecker' cd unison-cli cabal run --enable-profiling unison-cli-main:exe:unison -- +RTS -p ``` + +## Native compilation + +See the [readme](scheme-libs/racket/unison/Readme.md). diff --git a/scheme-libs/racket/unison/Readme.md b/scheme-libs/racket/unison/Readme.md index dafd7e3fa8..3984146df9 100644 --- a/scheme-libs/racket/unison/Readme.md +++ b/scheme-libs/racket/unison/Readme.md @@ -1,5 +1,18 @@ This directory contains libraries necessary for building and running -unison programs via Racket Scheme. +unison programs via Racket Scheme. The rough steps are as follows: + +* Build Racket libraries from the current Unison version. +* Build the `unison-runtime` binary. +* Pass the path to `unison-runtime` to `ucm`. + +Native compilation is done via the `compile.native` `ucm` command. +Under-the-hood, Unison does the following: + +* Convert the function to bytecode (similar to how `compile` command works). +* Call `unison-runtime` which will convert the bytecode to a temporary Racket + file. The Racket file is usually placed in your `.cache/unisonlanguage`. +* folder. Call `raco exe file.rkt -o executable` which will create a native + executable from the Racket source code. ## Prerequisites @@ -10,19 +23,56 @@ You'll need to have a couple things installed on your system: * [BLAKE2](https://github.com/BLAKE2/libb2) (you may need to install this manually) -In particular, our crypto functions require on both `libcrypto` (from openssl) and `libb2`. You may have to tell racket where to find `libb2`, by adding an entry to the hash table in your [`config.rktd` file](https://docs.racket-lang.org/raco/config-file.html). This is what I had, for an M1 mac w/ libb2 installed via Homebrew: +In particular, our crypto functions require on both `libcrypto` (from +openssl) and `libb2`. You may have to tell racket where to find `libb2`, +by adding an entry to the hash table in your +[`config.rktd` file](https://docs.racket-lang.org/raco/config-file.html). +This is what I had, for an M1 mac w/ `libb2` installed via Homebrew: ``` -(lib-search-dirs . (#f "/opt/homebrew/Cellar/libb2/0.98.1/lib/")) +$ cat scheme-libs/racket/config/config.rktd +#hash( + (lib-search-dirs . (#f "/opt/homebrew/Cellar/libb2/0.98.1/lib/")) +) ``` You'll also need to install `x509-lib` with `raco pkg install x509-lib` +Finally, some distributions only package `racket-minimal`. You'll need to +install the full compiler suite using `raco pkg install compiler-lib` +([source](https://www.dbrunner.de/blog/2016/01/12/using-racket-minimal-and-raco/)) + +## Building + +First, make sure unison is built (see [development](../../../development.markdown)) + +Next, use unison to generate the racket libraries. These are dependencies for +building `unison-runtime`. +* Read [gen-racket-libs.md](../../../../unison-src-transcripts-manual/gen-racket-libs.md). + It will contain two things: + * `ucm` and `unison` transcripts that generate the libraries + * Instructions on how to build `unison-runtime` using `raco` + +If everything went well you should now have a new executable in `scheme-libs/racket/unison-runtime`. +For example: +``` +$ file scheme-libs/racket/unison-runtime +scheme-libs/racket/unison-runtime: Mach-O 64-bit executable arm64 +``` ## Running the unison test suite -To run the test suite, first `stack build` (or `stack build --fast`), then: +Note that if you set up `config.rktd` above, you'll need to pass the path to its +folder in `PLTCONFIGDIR` before invoking unison or the test scripts: + +``` +export PLTCONFIGDIR=$(pwd)/scheme-libs/racket/config +``` + +If you don't, some of the tests will fail with eg `ffi-lib: could not load foreign library`. + +To run the test suite you can do: ``` -./unison-src/builtin-tests/jit-tests.sh $(stack exec which unison) --runtime-path +./unison-src/builtin-tests/jit-tests.sh $(stack exec which unison) --runtime-path scheme-libs/racket/unison-runtime ``` OR if you want to run the same tests in interpreted mode: @@ -31,7 +81,9 @@ OR if you want to run the same tests in interpreted mode: ./unison-src/builtin-tests/interpreter-tests.sh ``` -The above scripts fetch and cache a copy of base and the scheme-generating libraries, and copy this directory to `$XDG_DATA_DIRECTORY/unisonlanguage/scheme-libs`. +The above scripts fetch and cache a copy of base and the scheme-generating +libraries, and copy this directory to `$XDG_DATA_DIRECTORY/unisonlanguage/scheme-libs`. +Both scripts _should_ pass. ## Iterating more quickly diff --git a/unison-src/transcripts-manual/gen-racket-libs.md b/unison-src/transcripts-manual/gen-racket-libs.md index 178503c969..311d056641 100644 --- a/unison-src/transcripts-manual/gen-racket-libs.md +++ b/unison-src/transcripts-manual/gen-racket-libs.md @@ -20,12 +20,11 @@ complement of unison libraries for a given combination of ucm version and @unison/internal version. To set up racket to use these files, we need to create a package with -them. This is accomplished by running. +them. This is accomplished by running: - raco pkg install -t dir unison + raco pkg install -t dir scheme-libs/racket/unison -in the directory where the `unison` directory is located. Then the -runtime executable can be built with +After, the runtime executable can be built with raco exe scheme-libs/racket/unison-runtime.rkt From 05362b87d065015080c8096508bc7182d02a07c1 Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Sat, 13 Jul 2024 10:11:24 +0100 Subject: [PATCH 3/6] update command for integration tests in docs --- development.markdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/development.markdown b/development.markdown index fa5d613b84..d6d3eb1df0 100644 --- a/development.markdown +++ b/development.markdown @@ -42,7 +42,7 @@ Some tests are executables instead: * `stack exec transcripts` runs the transcripts-related integration tests, found in `unison-src/transcripts`. You can add more tests to this directory. * `stack exec transcripts -- prefix-of-filename` runs only transcript tests with a matching filename prefix. -* `stack exec integration-tests` runs the additional integration tests for cli. These tests are not triggered by `tests` or `transcripts`. +* `stack exec cli-integration-tests` runs the additional integration tests for cli. These tests are not triggered by `tests` or `transcripts`. * `stack exec unison -- transcript unison-src/transcripts-round-trip/main.md` runs the pretty-printing round trip tests * `stack exec unison -- transcript unison-src/transcripts-manual/benchmarks.md` runs the benchmark suite. Output goes in unison-src/transcripts-manual/benchmarks/output.txt. From 55560e58ca4c5f3ed881633e7b20dd2d5d16327b Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Thu, 18 Jul 2024 00:29:03 +0100 Subject: [PATCH 4/6] update transcripts-manual/gen-racket-libs.output.md --- .../gen-racket-libs.output.md | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/unison-src/transcripts-manual/gen-racket-libs.output.md b/unison-src/transcripts-manual/gen-racket-libs.output.md index 1e003ab489..178d4b6f4e 100644 --- a/unison-src/transcripts-manual/gen-racket-libs.output.md +++ b/unison-src/transcripts-manual/gen-racket-libs.output.md @@ -1,22 +1,21 @@ - When we start out, `./scheme-libs/racket` contains a bunch of library files that we'll need. They define the Unison builtins for Racket. Next, we'll download the jit project and generate a few Racket files from it. -```ucm +``` ucm jit-setup/main> lib.install @unison/internal/releases/0.0.18 - Downloaded 14917 entities. + Downloaded 14949 entities. I installed @unison/internal/releases/0.0.18 as unison_internal_0_0_18. ``` -```unison +``` unison go = generateSchemeBoot "scheme-libs/racket" ``` -```ucm +``` ucm Loading changes detected in scratch.u. @@ -29,7 +28,7 @@ go = generateSchemeBoot "scheme-libs/racket" go : '{IO, Exception} () ``` -```ucm +``` ucm jit-setup/main> run go () @@ -42,16 +41,23 @@ and @unison/internal version. To set up racket to use these files, we need to create a package with them. This is accomplished by running. - raco pkg install -t dir unison +``` +raco pkg install -t dir unison +``` -in the directory where the `unison directory is located. Then the +in the directory where the `unison` directory is located. Then the runtime executable can be built with - raco exe scheme-libs/racket/unison-runtime.rkt +``` +raco exe scheme-libs/racket/unison-runtime.rkt +``` and a distributable directory can be produced with: - raco distribute scheme-libs/racket/unison-runtime +``` +raco distribute scheme-libs/racket/unison-runtime +``` At that point, should contain the executable and all dependencies necessary to run it. + From 7ed45f6cd77b3d724503e30ff1050052b5ed6269 Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Thu, 18 Jul 2024 00:44:22 +0100 Subject: [PATCH 5/6] minor fixups --- scheme-libs/racket/unison/Readme.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scheme-libs/racket/unison/Readme.md b/scheme-libs/racket/unison/Readme.md index 3984146df9..f84d73e57f 100644 --- a/scheme-libs/racket/unison/Readme.md +++ b/scheme-libs/racket/unison/Readme.md @@ -25,8 +25,8 @@ You'll need to have a couple things installed on your system: In particular, our crypto functions require on both `libcrypto` (from openssl) and `libb2`. You may have to tell racket where to find `libb2`, -by adding an entry to the hash table in your -[`config.rktd` file](https://docs.racket-lang.org/raco/config-file.html). +by adding an entry to the hash table in your `config.rktd` +[file](https://docs.racket-lang.org/raco/config-file.html). This is what I had, for an M1 mac w/ `libb2` installed via Homebrew: ``` $ cat scheme-libs/racket/config/config.rktd @@ -46,7 +46,7 @@ First, make sure unison is built (see [development](../../../development.markdow Next, use unison to generate the racket libraries. These are dependencies for building `unison-runtime`. -* Read [gen-racket-libs.md](../../../../unison-src-transcripts-manual/gen-racket-libs.md). +* Read [gen-racket-libs.md](../../../unison-src/transcripts-manual/gen-racket-libs.md). It will contain two things: * `ucm` and `unison` transcripts that generate the libraries * Instructions on how to build `unison-runtime` using `raco` From c657e589240c511394b77caf6df8a36c6dd8c6bb Mon Sep 17 00:00:00 2001 From: Eduard Nicodei Date: Thu, 18 Jul 2024 00:50:16 +0100 Subject: [PATCH 6/6] remove extra word and mention libressl --- scheme-libs/racket/unison/Readme.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/scheme-libs/racket/unison/Readme.md b/scheme-libs/racket/unison/Readme.md index f84d73e57f..c8f7e76eb4 100644 --- a/scheme-libs/racket/unison/Readme.md +++ b/scheme-libs/racket/unison/Readme.md @@ -22,12 +22,11 @@ You'll need to have a couple things installed on your system: * [Racket](https://racket-lang.org/), with the executable `racket` on your path somewhere * [BLAKE2](https://github.com/BLAKE2/libb2) (you may need to install this manually) - -In particular, our crypto functions require on both `libcrypto` (from -openssl) and `libb2`. You may have to tell racket where to find `libb2`, -by adding an entry to the hash table in your `config.rktd` -[file](https://docs.racket-lang.org/raco/config-file.html). -This is what I had, for an M1 mac w/ `libb2` installed via Homebrew: +In particular, our crypto functions require both `libcrypto` (from openssl or +eg. libressl) and `libb2`. You may have to tell racket where to find `libb2`, by +adding an entry to the hash table in your +[`config.rktd` file](https://docs.racket-lang.org/raco/config-file.html). +This is what I had, for an M1 mac with `libb2` installed via Homebrew: ``` $ cat scheme-libs/racket/config/config.rktd #hash(