Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development image to test PopPunk branches #50

Merged
merged 29 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
054bc41
refactor: replace Dockerfile with separate production and development…
absternator Nov 27, 2024
e9acf2d
fix: update development image tags to include SHA for better versioning
absternator Nov 27, 2024
f5b8ab6
fix: streamline Dockerfile.dev by consolidating package installations…
absternator Nov 27, 2024
996b8bb
fix: update Dockerfile.dev to install Python 3.10
absternator Nov 27, 2024
ae8e61d
fix: correct Dockerfile.dev syntax by removing redundant RUN command
absternator Nov 27, 2024
180e907
fix: separate package installations in Dockerfile.dev for clarity and…
absternator Nov 27, 2024
a963269
fix: update mandrake installation in Dockerfile.dev to use classic so…
absternator Nov 27, 2024
5d05f6c
fix: correct typo in Dockerfile.dev for rapidnj installation solver
absternator Nov 27, 2024
a2aa01c
fet workimg verison
absternator Nov 29, 2024
f429068
fix: uncomment Dockerfile.dev commands for improved installation proc…
absternator Nov 29, 2024
35ed678
fix: update CMD in Dockerfile.dev to use array syntax for improved co…
absternator Nov 29, 2024
34e158f
fix: update Dockerfile.dev to use base environment and streamline com…
absternator Nov 29, 2024
a1ac4e7
docs: update README.md with deployment instructions and local develop…
absternator Nov 29, 2024
e0203a7
docs: update README.md and build scripts to include --with-dev flag f…
absternator Dec 2, 2024
912047a
docs: add note to README.md to remove --with-dev flag before merging PR
absternator Dec 2, 2024
a2dc8dd
fix: update CMD in Dockerfile.dev to use conda instead of mamba for c…
absternator Dec 3, 2024
55b5c50
fix: update README.md and Dockerfile.dev to use POPPUNK_VERSION varia…
absternator Dec 3, 2024
9fb3c5b
fix: simplify CMD in Dockerfile.dev by removing unnecessary bash invo…
absternator Dec 3, 2024
bfc77ac
fix: update CMD in Dockerfile.dev to use bash for command execution
absternator Dec 3, 2024
e22937c
fix: update CMD in Dockerfile.dev to use absolute path for bash
absternator Dec 3, 2024
fbfebb4
fix: update CMD in Dockerfile.dev to streamline conda command execution
absternator Dec 3, 2024
5cf6820
try github container registry
absternator Dec 3, 2024
0200a4b
fix: streamline GIT_SHA and GIT_BRANCH retrieval in docker/common
absternator Dec 3, 2024
b69e2a1
fix: update PACKAGE_ORG in docker/common to use bacpop
absternator Dec 3, 2024
cc62344
remove buildkte
absternator Dec 3, 2024
d84fb09
fix: update workflow name to reflect multiple docker images
absternator Dec 3, 2024
dfeeacb
fix: set CONDA_OVERRIDE_ARCHSPEC to ensure correct architecture for c…
absternator Dec 4, 2024
24f1611
fix: update branch pattern in workflow and improve README formatting
absternator Dec 5, 2024
f9b363b
fix: remove development flags from docker build and push commands
absternator Dec 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/build_and_push.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: build and push docker images
on:
push:
branches:
- main
pull_request:
branches:
- '*'
env:
BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Login to GHCR (GitHub Packages)
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build docker image
run: ./docker/build --with-dev # TODO: remove --with_dev before merging
- name: Push SHA docker image
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Push SHA docker image
- name: Push docker image

This pushes both branch and tag doesn't it? So this name is a bit misleading.

In some projects we delay pushing the branch tag until tests have passed - can't remember what we were doing in this one on buildkite.

run: ./docker/push --with-dev # TODO: remove --with_dev before merging
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,6 @@ You can build the image with `/docker/build --with-dev`, this new image can now

### Deployment

A pull request can be created so buildkite pushes the images to the docker hub. Add `--with-dev` to the build & push commands `pipeline.yaml`.
A pull request can be created so GHA pushes the images to the docker hub. Add `--with-dev` to the build & push commands `pipeline.yaml`.
**Ensure to remove the `--with-dev` flag before merging the PR.**
Then on the `beebop-deploy` the api image can be updated with the new dev image.
9 changes: 0 additions & 9 deletions buildkite/pipeline.yml

This file was deleted.

5 changes: 4 additions & 1 deletion docker/Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
FROM continuumio/miniconda3

# ensure conda does not install packages for the wrong architecture
ENV CONDA_OVERRIDE_ARCHSPEC=skylake

ARG POPPUNK_VERSION="v2.6.7"

# Make RUN commands use the new environment:
Expand Down Expand Up @@ -55,4 +58,4 @@ COPY . /beebop
WORKDIR /beebop
EXPOSE 5000

CMD ["bash", "-c", "conda run --no-capture-output -n base poetry run waitress-serve --port=5000 'beebop.app:app'"]
CMD ["conda", "run" ,"--no-capture-output", "-n", "base", "poetry", "run", "waitress-serve", "--port=5000", "beebop.app:app"]
30 changes: 11 additions & 19 deletions docker/common
Original file line number Diff line number Diff line change
@@ -1,32 +1,24 @@
#!/usr/bin/env bash
set -e

REGISTRY=ghcr.io
PACKAGE_ROOT=$(realpath $HERE/..)
PACKAGE_NAME=beebop-py
PACKAGE_ORG=mrcide
PACKAGE_ORG=bacpop
PACKAGE_DEV=dev

# Buildkite doesn't check out a full history from the remote (just the
# single commit) so you end up with a detached head and git rev-parse
# doesn't work
if [ "$BUILDKITE" = "true" ]; then
GIT_SHA=${BUILDKITE_COMMIT:0:7}
else
GIT_SHA=$(git -C "$PACKAGE_ROOT" rev-parse --short=7 HEAD)
fi

if [ "$BUILDKITE" = "true" ]; then
GIT_BRANCH=$BUILDKITE_BRANCH
GIT_SHA=$(git -C "$PACKAGE_ROOT" rev-parse --short=7 HEAD)
if [[ -v "BRANCH_NAME" ]]; then
GIT_BRANCH=${BRANCH_NAME}
else
GIT_BRANCH=$(git -C "$PACKAGE_ROOT" symbolic-ref --short HEAD)
GIT_BRANCH=$(git symbolic-ref --short HEAD)
fi

# production image
TAG_SHA="${PACKAGE_ORG}/${PACKAGE_NAME}:${GIT_SHA}"
TAG_BRANCH="${PACKAGE_ORG}/${PACKAGE_NAME}:${GIT_BRANCH}"
TAG_LATEST="${PACKAGE_ORG}/${PACKAGE_NAME}:latest"
TAG_SHA="${REGISTRY}/${PACKAGE_ORG}/${PACKAGE_NAME}:${GIT_SHA}"
TAG_BRANCH="${REGISTRY}/${PACKAGE_ORG}/${PACKAGE_NAME}:${GIT_BRANCH}"
TAG_LATEST="${REGISTRY}/${PACKAGE_ORG}/${PACKAGE_NAME}:latest"

# development image
TAG_DEV_SHA="${TAG_SHA}-${PACKAGE_DEV}"
TAG_DEV_BRANCH="${TAG_BRANCH}-${PACKAGE_DEV}"
TAG_DEV_SHA="${REGISTRY}/${TAG_SHA}-${PACKAGE_DEV}"
TAG_DEV_BRANCH="${REGISTRY}/${TAG_BRANCH}-${PACKAGE_DEV}"
POPPUNK_VERSION=v2.6.7 # can be version, branch or commit
1 change: 0 additions & 1 deletion docker/push
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ HERE=$(dirname $0)
. $HERE/common

# In case we switch agents between steps
[ ! -z $(docker images -q $TAG_SHA) ] || docker pull $TAG_SHA

docker tag $TAG_SHA $TAG_BRANCH
docker push $TAG_BRANCH
Expand Down
Loading