Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: enhance prep workflow, update the guide #228

Merged
merged 4 commits into from
Feb 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ jobs:
deno task prep:iroha --git $IROHA_GIT --git-rev $IROHA_REV
deno task prep:iroha:build

check:
needs: ['prep-crypto-wasm', 'prep-iroha']
check-only:
if: github.event_name == 'pull_request'
needs: [prep-iroha, prep-crypto-wasm]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand All @@ -83,7 +84,7 @@ jobs:
prep/crypto-wasm
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
deno-version: v2.2.x
- uses: actions/setup-node@v4
with:
node-version: 22
Expand All @@ -93,20 +94,11 @@ jobs:
npm install -g corepack@latest
echo "After : corepack version => $(corepack --version)"
corepack enable pnpm
- name: Deno install
run: deno task install
- name: Deno check
run: deno task check:all
- name: Deno lint, fmt
run: |
deno lint
deno fmt --check
- name: Test
run: deno task test
- run: deno task ok

publish:
check-and-publish:
if: github.event_name == 'push'
needs: [check]
needs: [prep-iroha, prep-crypto-wasm]
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -127,6 +119,14 @@ jobs:
prep/crypto-wasm
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: Publish
run: deno task publish
deno-version: v2.2.x
- uses: actions/setup-node@v4
with:
node-version: 22
- name: Enable pnpm via Corepack
run: |
echo "Before: corepack version => $(corepack --version || echo 'not installed')"
npm install -g corepack@latest
echo "After : corepack version => $(corepack --version)"
corepack enable pnpm
- run: deno task publish
83 changes: 35 additions & 48 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,71 +2,58 @@

This document explains how this repo works, and how to work with it.

Steps, in short:
## Pull Requests

1. Install Rust and `wasm-pack`.
2. Install Deno v2 and Node.js v22
3. Link Iroha repository
4. Explore `tasks` in the root `deno.json`
5. Work with the code
1. [Install the Deno CLI](https://docs.deno.com/runtime/manual/getting_started/installation).
2. [Install Node.js v22+](https://nodejs.org/en/download).
- And enable Corepack by `corepack enable` (making `pnpm` available).
3. Install [the Rust toolchain](https://rustup.rs/) and [`wasm-pack`](https://rustwasm.github.io/wasm-pack/installer/).
- Also make sure to install components: `rustup component add rust-src --target wasm32-unknown-unknown`.
4. Clone & fork this repository.
5. Link Iroha repository (see [below](#linking--building-iroha)).
6. Create a new branch for your changes.
7. Make your changes to the repo and ensure `deno task ok` passes successfully.
8. Commit your changes with clear messages, preferably following
[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
9. Submit a pull request.

TODO: setting up commit hooks
## Linking & Building Iroha

## Installing Rust and `wasm-pack`
For this project to function, you must link Iroha repository. There are two ways to do so.

```sh
# install rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# add necessary components
rustup component add rust-src --target wasm32-unknown-unknown
**Clone Iroha repository:**

# install wasm-pack
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
```sh
deno task prep:iroha --git https://github.com/hyperledger-iroha/iroha.git --git-rev v2.0.0-rc.1.0
```

## Installing Deno & Node.js

This must be pretty straightforward. A few notes though:

- While this project is mostly driven by Deno, there are some Node.js parts, unfortunately. Specifically,
`tests/browser` is a `pnpm` project, because otherwise it's hard to make Cypress work.
- Thus, run `corepack enable` (or install `npm i -g pnpm`) so that `pnpm` is also available.

## Linking & building Iroha

For this project to function, you must link Iroha repository. There are two ways to do so:
**Symlink to a local path:**

```sh
# symlink to the local path
deno task prep:iroha --path /path/to/local/iroha/clone
```

```sh
# clone the repo
deno task prep:iroha --git https://github.com/hyperledger-iroha/iroha.git --git-rev v2.0.0-rc.1.0
```
## Making Releases

After Iroha is linked, you need to prepare some artifacts from it (binaries such as `irohad`, `kagami`, `iroha_codec`;
`executor.wasm`; `schema.json`):
[`@deno/bump-workspaces`](https://github.com/denoland/bump-workspaces) could be of help:

```sh
deno task prep:iroha:build
```shell
deno run -A jsr:@deno/[email protected]/cli --dry-run
```

## Running tests
Guideline:

Please explore `tasks` in the root `deno.jsonc`.
1. Bump versions using the tool.
2. Submit a PR with the changes of the versions and `Releases.md`.
3. Merge, tag the commit in format `release-2025-02-20`, and push it.
4. Create a release on GitHub at this tag, using the new piece of `Releases.md` as a description and date (e.g.
"2025.02.20") as a title.

```sh
# run them all
deno task test
## Setting Up Commit Hooks _(optional)_

# unit, non-integration tests
deno run npm:vitest
# or
pnpm dlx vitest
```
You can add this to `.git/hooks/pre-commit`:

Tests are mostly written in Vitest (except the doctests), and it doesn't work very well with Deno (considering
migration). To improve development experience, consider running Vitest via `pnpm dlx` or similar.
```shell
#!/bin/sh
deno task ok
```
52 changes: 26 additions & 26 deletions deno.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -14,62 +14,62 @@
],
"tasks": {
"install": "deno install --allow-scripts=npm:cypress,npm:@parcel/watcher,npm:vue-demi",
"prep:iroha": "deno run --allow-all ./etc/task-prep-iroha.ts",
"prep:iroha:check": {
"description": "Ensure that Iroha is built",
"command": "deno task prep:iroha --check"
},
"prep:iroha:build": {
"description": "Build Iroha artifacts and produce `/prep/iroha` directory",
"command": "deno task prep:iroha --build"
"prep:iroha": {
"description": "Prepare Iroha. Must be manually linked first (via `--path` or `--git`)",
"command": "deno run -A ./etc/task-prep-iroha.ts"
},
"prep:crypto-wasm": {
"description": "Prepare crypto wasm and put into `@iroha/core`",
"description": "Prepare crypto wasm and put it into `@iroha/core`",
"command": "deno run -A ./etc/task-prep-crypto-wasm.ts"
},
"prep:crypto-wasm:copy": {
"description": "Only copy already built crypto wasm into `@iroha/core`",
"command": "deno task prep:crypto-wasm --copy"
},
"prep:codegen": {
"command": "deno run --allow-write --allow-read ./etc/task-codegen.ts",
"dependencies": ["prep:iroha:check"]
"command": "deno run -WR ./etc/task-codegen.ts",
"dependencies": ["prep:iroha"]
},
"prep:codegen:watch": "watchexec -e ts deno task prep:codegen",
"prep:ensure-ready": {
"dependencies": ["prep:codegen", "prep:crypto-wasm:copy", "prep:iroha:check"]
"prep:ok": {
"dependencies": ["prep:codegen", "prep:crypto-wasm", "prep:iroha"]
},
"check:all": {
"check": {
"command": "deno check --doc .",
"dependencies": ["prep:ensure-ready"]
"dependencies": ["prep:ok"]
},
"test:deno": {
"command": "deno test --doc",
"dependencies": ["prep:ok"]
},
"test:deno": "deno test --doc --allow-read",
"test:vitest": {
"dependencies": ["prep:ensure-ready"],
"dependencies": ["prep:ok"],
"description": "Run Vitest",
"command": "vitest run"
},
"test:integration:node": {
"dependencies": ["prep:ensure-ready"],
"dependencies": ["prep:ok"],
"command": "cd tests/node && deno task test"
},
"test:integration:browser": {
"dependencies": ["prep:ensure-ready"],
"dependencies": ["prep:ok"],
"command": "cd tests/browser && deno task test"
},
"test": {
"description": "Run all tests, from unit to integration",
"command": "deno task test:deno && deno task test:vitest && deno task test:integration:node && deno task test:integration:browser"
},
"dev:run-test-peer": {
"dependencies": ["prep:ensure-ready"],
"description": "Run an Iroha peer with test configuration. Could be useful for development",
"dependencies": ["prep:ok"],
"command": "cd tests/support/test-peer && deno task run"
},
"ok": {
"description": "Run all checks",
"dependencies": ["prep:ok", "install"],
"command": "deno lint && deno fmt --check && deno task check && deno task test"
},
"publish": {
// https://github.com/denoland/deno/issues/28096
"description": "Publish workspace packages",
"command": "deno run --allow-read --allow-run --allow-env ./etc/task-publish.ts",
"dependencies": ["check:all"]
"command": "deno run -A ./etc/task-publish.ts",
"dependencies": ["ok"]
}
},
"fmt": {
Expand Down
7 changes: 6 additions & 1 deletion etc/task-codegen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,12 @@ import * as dprintTS from 'npm:@dprint/typescript'
import * as colors from '@std/fmt/colors'

const tsFormatter = dprint.createFromBuffer(await Deno.readFile(dprintTS.getPath()))
const formatTS = (code: string) => tsFormatter.formatText({ filePath: 'file.ts', fileText: code })
const formatTS = (code: string) => {
console.time('dprint')
const text = tsFormatter.formatText({ filePath: 'file.ts', fileText: code })
console.timeEnd('dprint')
return text
}

async function write({ file, code }: { file: string; code: string }) {
let status: string
Expand Down
38 changes: 27 additions & 11 deletions etc/task-prep-crypto-wasm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import $ from 'jsr:@david/dax'
import * as colors from '@std/fmt/colors'
import { pathRel, resolveFromRoot } from './util.ts'
import { copy, emptyDir } from 'jsr:@std/fs'
import { assert } from '@std/assert/assert'
import { assertEquals } from '@std/assert/equals'

const PROJECT_DIR = resolveFromRoot('crypto-wasm')
const PREP_DIR = resolveFromRoot('prep/crypto-wasm')
Expand All @@ -12,37 +14,51 @@ const CRATE_NAME = 'iroha_crypto_wasm'

async function buildWasm() {
const outDir = PREP_DIR
// TODO: empty dir?
console.log(' ' + colors.yellow(`empty ${pathRel(PREP_DIR)}`))
await emptyDir(PREP_DIR)

$.logStep(`Running @deno/wasmbuild...`)
$.logStep(`Building with @deno/wasmbuild...`)
await $`deno run -A jsr:@deno/wasmbuild --out ${outDir}`.cwd(PROJECT_DIR)
console.log(` ${colors.yellow(`wasmbuild ready at ${pathRel(outDir)}`)}'`)
console.log(` ${colors.yellow(`write ${pathRel(outDir)}`)}`)
}

async function checkBuildReady() {
try {
const files = new Set<string>()
for await (const i of Deno.readDir(PREP_DIR)) {
assert(i.isFile)
files.add(i.name)
}
assertEquals(files, new Set([`.d.ts`, '.internal.js', '.js', '.wasm'].map((x) => `${CRATE_NAME}${x}`)))
return true
} catch (err) {
$.logWarn('Error whiule checking build artifacts:', err)
return false
}
}

async function copyOutputs() {
const targetDir = resolveFromRoot('packages/core/crypto/wasm')
const files = [`${CRATE_NAME}.d.ts`, `${CRATE_NAME}.wasm`, `${CRATE_NAME}.internal.js`]

console.log(' ' + colors.yellow(`empty ${pathRel(targetDir)}`))
await emptyDir(targetDir)

for (const i of files) {
const src = resolveFromRoot(PREP_DIR, i)
const dest = path.join(targetDir, i)
console.log(` ${colors.yellow(`write ${pathRel(dest)}`)}`)
try {
await copy(src, dest)
} catch (err) {
$.logError(`Failed to copy. Is ${PREP_DIR} ready?`)
throw err
}
await copy(src, dest)
}
}

const args = parseArgs(Deno.args, {
boolean: ['copy'],
boolean: ['force'],
})

if (!args.copy) {
if (!args.force && (await checkBuildReady())) {
$.logStep(`Skipping build step (override with ${colors.cyan('--force')})`)
} else {
await buildWasm()
}
await copyOutputs()
Loading