Skip to content

Commit

Permalink
Improve local dev DX by using just (#501)
Browse files Browse the repository at this point in the history
* Replace the `Makefile` with a `justfile`
* Update CI jobs to use it
* Update `CONTRIBUTING.md`
* Bump `pip` deps
  • Loading branch information
Blacksmoke16 authored Jan 25, 2025
1 parent 178fc15 commit 8c7e9ac
Show file tree
Hide file tree
Showing 7 changed files with 173 additions and 74 deletions.
12 changes: 8 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,24 +31,26 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: extractions/setup-just@v2
- name: Install Crystal
uses: crystal-lang/install-crystal@v1
with:
crystal: ${{ matrix.crystal }}
- name: Check Format
run: crystal tool format --check
run: just format
coding_standards:
runs-on: ubuntu-latest
container:
image: crystallang/crystal:latest
steps:
- uses: actions/checkout@v4
- uses: extractions/setup-just@v2
- name: Install Dependencies
run: shards install
env:
SHARDS_OVERRIDE: shard.dev.yml
- name: Ameba
run: ./bin/ameba
run: just ameba
test_compiled:
strategy:
fail-fast: false
Expand All @@ -59,6 +61,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: extractions/setup-just@v2
- name: Install kcov
if: matrix.crystal == 'nightly'
run: |
Expand All @@ -82,7 +85,7 @@ jobs:
env:
SHARDS_OVERRIDE: shard.dev.yml
- name: Compiled Specs
run: ./scripts/test.sh all compiled
run: just test-compiled
shell: bash
- uses: codecov/codecov-action@v5
if: matrix.crystal == 'nightly' && github.event_name != 'schedule' # Only want to upload coverage report once in the matrix
Expand Down Expand Up @@ -121,6 +124,7 @@ jobs:
if: github.event_name == 'pull_request'
with:
fetch-depth: 0
- uses: extractions/setup-just@v2
- name: Install kcov
if: matrix.os == 'ubuntu-latest' && matrix.crystal == 'nightly'
run: |
Expand All @@ -144,7 +148,7 @@ jobs:
env:
SHARDS_OVERRIDE: shard.dev.yml
- name: Specs
run: ./scripts/test.sh all unit
run: just test-unit
shell: bash
- uses: codecov/codecov-action@v5
if: matrix.os == 'ubuntu-latest' && matrix.crystal == 'nightly' && github.event_name != 'schedule' # Only want to upload coverage report once in the matrix
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ jobs:
with:
ssh-private-key: ${{ secrets.SSH_KEY }}
- uses: actions/checkout@v4
- uses: extractions/setup-just@v2
- name: Install Crystal
uses: crystal-lang/install-crystal@v1
- name: Install Components
Expand All @@ -23,7 +24,7 @@ jobs:
with:
python-version: 3.x
- name: Build Docs
run: make build
run: just build-docs
- name: Publish to Cloudflare Pages
uses: cloudflare/wrangler-action@v3
with:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
with:
ssh-private-key: ${{ secrets.SSH_KEY }}
- uses: actions/checkout@v4
- uses: extractions/setup-just@v2
- name: Install Crystal
uses: crystal-lang/install-crystal@v1
- name: Install Components
Expand All @@ -35,7 +36,7 @@ jobs:
with:
python-version: 3.x
- name: Build Docs
run: make build
run: just build-docs
- name: Publish to Cloudflare Pages
uses: cloudflare/wrangler-action@v3
with:
Expand Down
59 changes: 42 additions & 17 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,34 +16,59 @@ The [issue tracker](https://github.com/athena-framework/athena/issues) is the he

Please always **open a new issue before sending a pull request** if you want to add a new feature to Athena, unless it is a minor obvious fix, or is in relation to an already open & approved issue. This reduces the likelihood of wasted effort, and ensures the end result is robust by being able to work out implementation details _before_ the work is started.

## Testing
## Local Development

Due to Athena's usage of a monorepo, testing is handled slightly differently than with a normal shard. Mainly that all testing is done directly from the root of the monorepo itself. Athena's `./scripts/test.sh` helper script can be used to make this easier by providing a singular entrypoint that defines all of the common options and flags needed to run the tests for all, or a single component. However, before it can be used the components themselves need installed.
Due to Athena's usage of a monorepo, the same single repo can be used to contribute code to all components.

Shards are ideally installed via symlink, which allows for updates to be instantly available for testing without having to juggle branches/edit each component's `shard.yml`. The `shard.dev.yml` file can be used in conjunction with the `SHARDS_OVERRIDE` environmental variable for this purpose:
In addition to Crystal itself, Athena makes use of [just](https://just.systems/man/en/introduction.html) as its command runner.
`just` provides a simple way of executing common commands needed for development.

Once you have it installed, and have cloned the monorepo, first install all the shard dependencies by running:

```sh
$ SHARDS_OVERRIDE=shard.dev.yml shards update
just install
```

From here, the tests for a specific component may be ran via the helper script:
And that's it, you are now ready to start coding!
From here there are some additional optional tools that will come in handy:

```sh
$ ./scripts/test.sh routing
```
1. [typos](https://github.com/crate-ci/typos) - Source code spell checker, used as part of the `spellcheck` recipe.
1. [watchexec](https://github.com/watchexec/watchexec) - Executes commands in response to file modifications, used as part of the `watch` and `watch-spec` recipes.
1. [kcov](https://github.com/SimonKagstrom/kcov) - Code coverage tool, used to generate coverage reports/files as part of the `test` recipes.
1. [python](https://www.python.org/) - The programming language, used for the `docs` related recipes.

Or alternatively, for all components:
**TIP:** Running `just` will provide a summary of available recipes.

```sh
$ ./scripts/test.sh
```
### Development

Athena also leverages [ameba](https://github.com/crystal-ameba/ameba) as its form of static code analysis. It too can be ran directly from the root of the monorepo after the required shards are installed:
Because of Athena's usage of a monorepo some interactions may be different than a normal shard.
Mainly that most things can be done from the root of the repo; no need to `cd` to whatever component you're working on, and need to go through `just`.

```sh
$ ./bin/ameba
```
For exploratory work, the suggested workflow is to have your code in the related component's entry point file.
E.g. `src/components/clock/src/athena-clock.cr` for the `clock` component.
From here you can run `just watch clock` and that will re-run the file when changes are made.
This makes it simple to play around with early implementations before there is proper test coverage.

#### Testing

### Athena Spec
Similar to development itself, running the specs are also done through `just`: `just test clock` would run the spec suite for that component, and generate coverage information if you have `kcov` installed.
The `watch-test` recipe can come in handy to provide quicker feedback while the tests are under development.

##### Athena Spec

Many Athena components make use of [Athena Spec](https://athenaframework.org/Spec/) for their unit/integration tests. This library provides an alternate DSL that is 100% compatible with the standard library's `Spec` module. I.e. they can be used together seamlessly, using whatever DSL is more appropriate for what is being tested. Being familiar with the base [ASPEC::TestCase](https://athenaframework.org/Spec/TestCase/) type will not only make reading the specs easier, but writing them as well. It comes with various features to make the tests simpler, reusable, and extensible. You may even want to use it in your own projects :wink:.

### Linting

Beyond testing, Athena makes use of various forms of linting, including:

* [ameba](https://github.com/crystal-ameba/ameba) for static code analysis
* [typos](https://github.com/crate-ci/typos) for spell checking
* The Crystal [formatter](https://crystal-lang.org/reference/guides/writing_shards.html#coding-style) for code formatting

All of these can be executed at once via the `just lint` recipe, but may also be ran individual as needed via their related `just` recipe.

### Documentation

Athena's [documentation](https://athenaframework.org/) site may be built locally via the `just build-docs` recipe.
Alternatively, a live-updating server may be started via the `just serve-docs` recipe.
41 changes: 0 additions & 41 deletions Makefile

This file was deleted.

109 changes: 109 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Configuration

OUTPUT_DIR := './site'

# Binaries
# Scoped to the justfile so do not need to be exported

MKDOCS := './.venv/bin/mkdocs'
PIP := './.venv/bin/pip3'
PIP_COMPILE := './.venv/bin/pip-compile'

# Needs to be exported so that the `spec` component can pick up on the customized $CRYSTAL env var.

export CRYSTAL := 'crystal'

# Ensure when MkDocs is generating the docs for each component it's able to find their shard dependencies
# via the monorepo's `lib/` dir versus `src/components/<name>/lib` which doesn't exist.

export CRYSTAL_PATH := (invocation_directory_native() / ('lib' + if os() == 'windows' { ';' } else { ':' })) + shell('$1 env CRYSTAL_PATH', CRYSTAL)

_default:
@just --list --unsorted

# Installs dev dependencies
[group('dev')]
install:
SHARDS_OVERRIDE=shard.dev.yml shards update

# Runs and watches for changes to the main entrypoint file of the provided `component`
[group('dev')]
watch component:
watchexec --restart --watch=src/ --emit-events-to=none --clear -- {{ CRYSTAL }} run src/components/{{ component }}/src/{{ if component == 'framework' { 'athena' } else { 'athena-' + component } }}.cr

# Runs the test suite of the provided `component`, or `all` for all components, and watches for changes
[group('dev')]
watch-test component:
watchexec --restart --watch=src/ --emit-events-to=none --clear -- {{ CRYSTAL }} spec src/components/{{ component }}/

# Runs the test suite of the provided `component`, defaulting to `all` components
[group('dev')]
test component='all':
./scripts/test.sh {{ component }}

# Runs the unit test suite of the provided `component`, defaulting to `all` components
[group('dev')]
test-unit component='all':
./scripts/test.sh {{ component }} unit

# Runs the compiled test suite of the provided `component`, defaulting to `all` components
[group('dev')]
test-compiled component='all':
./scripts/test.sh {{ component }} compiled

# Runs all check related tasks
[group('check')]
lint: spellcheck format ameba

# Runs the Crystal formatter
[group('check')]
format:
{{ CRYSTAL }} tool format

# Runs the Crystal formatter, fixing any issues
[group('check')]
format-fix:
{{ CRYSTAL }} tool format --fix

# Runs `Ameba` static analysis
[group('check')]
ameba:
./bin/ameba

# Runs `typos` spellchecker
[group('check')]
spellcheck:
typos

# Build the docs
[group('docs')]
build-docs: _mkdocs
{{ MKDOCS }} build -d {{ OUTPUT_DIR }}

# Serve live-preview of the docs
[group('docs')]
serve-docs: _mkdocs
{{ MKDOCS }} serve --open

# Clean MKDocs build artifacts
[group('docs')]
clean-docs:
rm -rf {{ OUTPUT_DIR }}
find src/components -type d -name "site" -exec rm -rf {} +

# Upgrade python deps
[group('administrative')]
upgrade: _pip
{{ PIP_COMPILE }} -U requirements.in

# Clean build artifacts (.venv), and docs
[group('administrative')]
clean: clean-docs
rm -rf .venv

_pip:
python3 -m venv .venv
{{ PIP }} install -q pip-tools

_mkdocs: _pip
{{ PIP }} install -q -r requirements.txt
20 changes: 10 additions & 10 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile requirements.in
#
babel==2.16.0
# via mkdocs-material
certifi==2024.8.30
certifi==2024.12.14
# via requests
charset-normalizer==3.4.0
charset-normalizer==3.4.1
# via requests
click==8.1.7
click==8.1.8
# via
# mkdocs
# mkdocstrings
Expand All @@ -20,7 +20,7 @@ ghp-import==2.1.0
# via mkdocs
idna==3.10
# via requests
jinja2==3.1.4
jinja2==3.1.5
# via
# mkdocs
# mkdocs-material
Expand Down Expand Up @@ -57,7 +57,7 @@ mkdocs==1.6.1
# mkdocs-material
# mkdocs-section-index
# mkdocstrings
mkdocs-autorefs==1.2.0
mkdocs-autorefs==1.3.0
# via
# mkdocstrings
# mkdocstrings-crystal
Expand Down Expand Up @@ -87,9 +87,9 @@ platformdirs==4.3.6
# via
# mkdocs-get-deps
# mkdocstrings
pygments==2.18.0
pygments==2.19.1
# via mkdocs-material
pymdown-extensions==10.12
pymdown-extensions==10.14.1
# via
# mkdocs-material
# mkdocstrings
Expand All @@ -107,9 +107,9 @@ regex==2024.11.6
# via mkdocs-material
requests==2.32.3
# via mkdocs-material
six==1.16.0
six==1.17.0
# via python-dateutil
urllib3==2.2.3
urllib3==2.3.0
# via requests
watchdog==6.0.0
# via mkdocs

0 comments on commit 8c7e9ac

Please sign in to comment.