Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split and parralelize CI jobs #74

Closed
wants to merge 8 commits into from
Closed

Split and parralelize CI jobs #74

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c983201
Split CI into 6 separate jobs
browniebroke Feb 16, 2025
eaf17c9
Split CI build/pull & run into separate steps
browniebroke Feb 16, 2025
e81804e
Separate build and push in 2 jobs
browniebroke Feb 18, 2025
469390c
Add docker build cache
browniebroke Feb 18, 2025
a1ea454
Build and push using official actions
browniebroke Feb 18, 2025
f2982ab
Bypass GH container registry and use upload/download-artifact
browniebroke Feb 18, 2025
417c665
Only pull missing images
browniebroke Feb 18, 2025
cd26271
Revert back to GH container registry
browniebroke Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions .github/workflows/check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,48 @@ name: Check
on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v2

- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and push
uses: docker/build-push-action@v6
with:
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/django-pg-zero-downtime-migrations:latest

check:
name: run checks
needs:
- build
strategy:
fail-fast: false
matrix:
tox-filter:
- "py3.8"
- "py3.9"
- "py3.10"
- "py3.11"
- "py3.12"
- "py3.13"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea, however it solve issue partially as we still has different django, psycopg and postgres versions. Is it possible to move tox matrix that include python, django, psycopg, postgres versions fully to github actions matrix?

Copy link
Contributor Author

@browniebroke browniebroke Feb 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it's true that it's a partial solution. It's a trade-off between CI workflow configuration complexity and convenience. If the split is fine grained down to 1 CI job per tox env, then your logs are super clear, but the YAML can get quite complicated.

I find that having 1 job per Python version is usually a good balance... I know of a few Django packages I use doing that (whitenoise, django-cors-headers) and that's the approach in the @django-commons org as well.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, your approach anyway better that current

name: run checks ${{ matrix.tox-filter }}
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v2
- name: pull images
run: docker compose pull --quiet
- name: run checks
run: docker compose run --rm django-pg-zero-downtime-migrations-tests tox
run: docker compose run --rm django-pg-zero-downtime-migrations-tests tox -f ${{ matrix.tox-filter }}
1 change: 1 addition & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ services:
- ./docker_postgres_init.sql:/docker-entrypoint-initdb.d/docker_postgres_init.sql

django-pg-zero-downtime-migrations-tests:
image: ghcr.io/${GITHUB_REPOSITORY_OWNER:-tbicr}/django-pg-zero-downtime-migrations:latest
build: .
depends_on:
- pg17
Expand Down
Loading