Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 3.21 CI #672

Merged
merged 19 commits into from
Oct 9, 2024
Merged

Fix 3.21 CI #672

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
34ed99b
Use new text fixture signing keys
gerrod3 Oct 8, 2024
1f62b4c
Update workflow actions to latest
gerrod3 Oct 8, 2024
44140e6
Use correct version of pulp-cli during tests
markafarrell Oct 8, 2024
1d8c61d
Reassign user pulp as owner of the DB
mdellweg Oct 8, 2024
52ea4e6
Hardcode compose images to 3.21
gerrod3 Oct 8, 2024
f8be370
Remove installing pulpcore in non-s6 base image
gerrod3 Oct 8, 2024
dc2440d
Add some extra build dependencies to non-s6 base image
gerrod3 Oct 8, 2024
eb1b604
Add more missing dependencies to non-s6 base image
gerrod3 Oct 8, 2024
e487fc4
Special handling for non-s6 galaxy install
gerrod3 Oct 8, 2024
affd6f2
Add md1 to ALLOWED_CHECKSUMS for tests
gerrod3 Oct 8, 2024
290eb97
Correct the order on the non-s6 galaxy installs
gerrod3 Oct 8, 2024
428a7dd
Lock down plugin versions to last viable Y release
gerrod3 Oct 8, 2024
46b6615
Streamline s6 image CI
gerrod3 Oct 8, 2024
28f917d
Pip install createrepo_c for non-s6 base image
gerrod3 Oct 8, 2024
520d9a0
Proper quotes for build args & proper user for collectstatic
gerrod3 Oct 8, 2024
84ceaee
Remove createrepo_c rpm dependencies
gerrod3 Oct 8, 2024
b6db2c0
Turn off verify-ssl for cli tests
gerrod3 Oct 8, 2024
bbe6900
Refactor s6 publish workflow
gerrod3 Oct 8, 2024
2327388
Use correct build-arg for s6 images
gerrod3 Oct 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 22 additions & 48 deletions .github/workflows/pr_build_images.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,13 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
# by default, it uses a depth of 1
# this fetches all history so that we can read each commit
fetch-depth: 0
- name: Set up Python 3.8
uses: actions/setup-python@v1
with:
python-version: '3.8'
- uses: actions/setup-python@v4

- name: Check commit message
if: github.event_name == 'pull_request'
env:
Expand All @@ -45,49 +43,22 @@ jobs:
fail-fast: false
matrix:
include:
- ci_type: pulp_ci_centos
ci_image: pulp-ci-centos
test_type: pulp_galaxy_ng
test_image: pulp-galaxy-ng
- ci_type: pulp_ci_centos
ci_image: pulp-ci-centos
test_type: pulp
test_image: pulp
- test_image: pulp-galaxy-ng
tag: "4.6"
- test_image: pulp
tag: "3.21"
- test_image: pulp-galaxy-ng
tag: "4.6-https"
- test_image: pulp
tag: "3.21-https"
steps:
- uses: actions/checkout@v2
- name: Build the images
- uses: actions/checkout@v4
- name: Set https build args
if: contains(fromJSON('["4.6-https", "3.21-https"]'), matrix.tag)
run: |
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-x86_64.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-noarch.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-arch.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-noarch.tar.xz
unxz s6-overlay*.tar.xz
gzip s6-overlay*.tar
docker build --file s6_images/${{ matrix.ci_type }}/Containerfile --tag pulp/${{ matrix.ci_image }}:latest .
docker build --file s6_images/${{ matrix.test_type }}/Containerfile --tag pulp/${{ matrix.test_image }}:latest .
- name: Test the images
run: s6_images/assets/test.sh "pulp/${{ matrix.test_image }}:latest"
- name: Display log on error
if: failure()
run: docker logs pulp
echo "BUILD_ARG=--build-arg SCHEME=https" >> $GITHUB_ENV
echo "TEST_ARG=https" >> $GITHUB_ENV

sslbuild:
runs-on: ubuntu-latest
needs: lint
strategy:
fail-fast: false
matrix:
include:
- ci_type: pulp_ci_centos
ci_image: pulp-ci-centos
test_type: pulp_galaxy_ng
test_image: pulp-galaxy-ng
- ci_type: pulp_ci_centos
ci_image: pulp-ci-centos
test_type: pulp
test_image: pulp
steps:
- uses: actions/checkout@v2
- name: Build the images
run: |
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-x86_64.tar.xz
Expand All @@ -96,10 +67,13 @@ jobs:
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-noarch.tar.xz
unxz s6-overlay*.tar.xz
gzip s6-overlay*.tar
docker build --build-arg SCHEME="https" --file s6_images/${{ matrix.ci_type }}/Containerfile --tag pulp/${{ matrix.ci_image }}:https .
docker build --build-arg FROM_TAG="https" --file s6_images/${{ matrix.test_type }}/Containerfile --tag pulp/${{ matrix.test_image }}:https .
LOCATION="${{ matrix.test_image }}"
docker build --file s6_images/pulp_ci_centos/Containerfile --tag pulp/pulp-ci-centos:${{ matrix.tag }} . ${{ env.BUILD_ARG }}
docker build --file "s6_images/${LOCATION//-/_}/Containerfile" --tag pulp/${{ matrix.test_image }}:${{ matrix.tag }} . --build-arg FROM_TAG=${{ matrix.tag }}

- name: Test the images
run: s6_images/assets/test.sh "pulp/${{ matrix.test_image }}:https" https
run: s6_images/assets/test.sh "pulp/${{ matrix.test_image }}:${{ matrix.tag }}" ${{ env.TEST_ARG }}

- name: Display log on error
if: failure()
run: docker logs pulp
200 changes: 41 additions & 159 deletions .github/workflows/publish_images.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,7 @@ name: Publish OCI Images
on:
push:
branches:
- latest
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 1 * * *'
- 3.21
workflow_dispatch:

env:
Expand All @@ -17,8 +14,21 @@ env:
jobs:
publish:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- image: pulp-galaxy-ng
tag: "4.6"
- image: pulp
tag: "3.21"
- image: pulp-galaxy-ng
tag: "4.6-https"
- image: pulp
tag: "3.21-https"
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- name: Download s6-overlay
run: |
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-x86_64.tar.xz
Expand All @@ -27,159 +37,39 @@ jobs:
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-noarch.tar.xz
unxz s6-overlay*.tar.xz
gzip s6-overlay*.tar
- name: Set version
- name: Set build args
if: contains(fromJSON('["4.6-https", "3.21-https"]'), matrix.tag)
run: |
version="${GITHUB_REF#refs/heads/}"
echo "Building and publishing $version."
echo "VERSION=$version" >> $GITHUB_ENV

- uses: actions/setup-python@v3
with:
python-version: "3.8"

- name: Install python dependencies
if: github.event_name == 'schedule'
run: |
echo ::group::PYDEPS
pip install gitpython requests packaging jinja2 pyyaml
echo ::endgroup::

- name: Dispatching
if: github.event_name == 'schedule'
run: python .ci/scripts/update_ci_branches.py
env:
GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
echo "BUILD_ARG=--build-arg SCHEME=https" >> $GITHUB_ENV
echo "TEST_ARG=https" >> $GITHUB_ENV

- name: Build the images
run: |
docker build --file s6_images/pulp_ci_centos/Containerfile --tag pulp/pulp-ci-centos:latest .
docker tag pulp/pulp-ci-centos:latest ghcr.io/pulp/pulp-ci-centos:latest
docker tag pulp/pulp-ci-centos:latest quay.io/pulp/pulp-ci-centos:latest
if [ "$VERSION" = "latest" ]; then
docker build --file s6_images/pulp_galaxy_ng/Containerfile --tag pulp/pulp-galaxy-ng:latest .
docker tag pulp/pulp-galaxy-ng:latest ghcr.io/pulp/pulp-galaxy-ng:latest
docker tag pulp/pulp-galaxy-ng:latest quay.io/pulp/all-in-one-galaxy:latest
docker build --file s6_images/pulp_ci_centos/Containerfile --tag pulp/pulp-ci-centos:${{ matrix.tag }} . ${{ env.BUILD_ARG }}
docker tag pulp/pulp-ci-centos:${{ matrix.tag }} ghcr.io/pulp/pulp-ci-centos:${{ matrix.tag }}
docker tag pulp/pulp-ci-centos:${{ matrix.tag }} quay.io/pulp/pulp-ci-centos:${{ matrix.tag }}

LOCATION="${{ matrix.image }}"
IMAGE=${LOCATION//?(pulp-|-ng)/}
LOCATION=${LOCATION//-/_}
docker build --file "s6_images/${LOCATION}/Containerfile" --tag pulp/pulp-galaxy-ng:${{ matrix.tag }} . --build-arg FROM_TAG=${{ matrix.tag }}
docker tag pulp/${{ matrix.image }}:${{ matrix.tag }} ghcr.io/pulp/${{ matrix.image }}:${{ matrix.tag }}
docker tag pulp/${{ matrix.image }}:${{ matrix.tag }} "quay.io/pulp/all-in-one-${IMAGE}:${{ matrix.tag }}"
echo "IMAGE=$IMAGE" >> $GITHUB_ENV

# Be resilient if this fails
docker build --file s6_images/pulp_nightly/Containerfile --tag pulp/pulp:nightly . || true
docker tag pulp/pulp:nightly ghcr.io/pulp/pulp:nightly || true
docker tag pulp/pulp:nightly quay.io/pulp/all-in-one-pulp:nightly || true
else
docker tag pulp/pulp-ci-centos:latest pulp/pulp-ci-centos:$VERSION
docker tag pulp/pulp-ci-centos:latest ghcr.io/pulp/pulp-ci-centos:$VERSION
docker tag pulp/pulp-ci-centos:latest quay.io/pulp/pulp-ci-centos:$VERSION
fi
docker build --file s6_images/pulp/Containerfile --tag pulp/pulp:$VERSION .
docker tag pulp/pulp:$VERSION ghcr.io/pulp/pulp:$VERSION
docker tag pulp/pulp:$VERSION quay.io/pulp/all-in-one-pulp:$VERSION
- name: Test the images
run: s6_images/assets/test.sh "pulp/pulp:$VERSION"
run: s6_images/assets/test.sh "pulp/${{ matrix.image }}:${{ matrix.tag }}" ${{ env.TEST_ARG }}

- name: Github login
env:
PULP_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PULP_GITHUB_USERNAME: ${{ github.actor }}
run: echo "$PULP_GITHUB_TOKEN" | docker login -u "$PULP_GITHUB_USERNAME" --password-stdin ghcr.io
- name: Push ci image to github
run: |
docker push ghcr.io/pulp/pulp:$VERSION
docker push ghcr.io/pulp/pulp-ci-centos:$VERSION
if [ "$VERSION" = "latest" ]; then
docker push ghcr.io/pulp/pulp-galaxy-ng:latest
# Be resilient if this fails
docker push ghcr.io/pulp/pulp:nightly || true
fi
- name: Docker login
env:
DOCKER_BOT_PASSWORD: ${{ secrets.DOCKER_BOT_PASSWORD }}
DOCKER_BOT_USERNAME: ${{ secrets.DOCKER_BOT_USERNAME }}
run: echo "$DOCKER_BOT_PASSWORD" | docker login -u "$DOCKER_BOT_USERNAME" --password-stdin docker.io
- name: Push ci image to dockerhub
run: |
docker push docker.io/pulp/pulp:$VERSION
docker push docker.io/pulp/pulp-ci-centos:$VERSION
if [ "$VERSION" = "latest" ]; then
docker push docker.io/pulp/pulp-galaxy-ng:latest
# Be resilient if this fails
docker push docker.io/pulp/pulp:nightly || true
fi
- name: Quay login
env:
QUAY_BOT_PASSWORD: ${{ secrets.QUAY_BOT_PASSWORD }}
QUAY_BOT_USERNAME: ${{ secrets.QUAY_BOT_USERNAME }}
run: echo "$QUAY_BOT_PASSWORD" | docker login -u "$QUAY_BOT_USERNAME" --password-stdin quay.io
- name: Push ci image to quay
run: |
docker push quay.io/pulp/all-in-one-pulp:$VERSION
docker push quay.io/pulp/pulp-ci-centos:$VERSION
if [ "$VERSION" = "latest" ]; then
docker push quay.io/pulp/all-in-one-galaxy:latest
# Let this be the last thing so it can indicate failure without breaking anything else
docker push quay.io/pulp/all-in-one-pulp:nightly
fi
- name: Display log on error
if: failure()
run: docker logs pulp

ssl_publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Download s6-overlay
run: |
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-x86_64.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-noarch.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-arch.tar.xz
wget https://github.com/just-containers/s6-overlay/releases/download/v3.1.2.1/s6-overlay-symlinks-noarch.tar.xz
unxz s6-overlay*.tar.xz
gzip s6-overlay*.tar
- name: Set version
run: |
version="${GITHUB_REF#refs/heads/}-https"
echo "Building and publishing $version."
echo "VERSION=$version" >> $GITHUB_ENV
- name: Build the images
run: |
docker build --build-arg SCHEME="https" --file s6_images/pulp_ci_centos/Containerfile --tag pulp/pulp-ci-centos:https .
docker tag pulp/pulp-ci-centos:https ghcr.io/pulp/pulp-ci-centos:https
docker tag pulp/pulp-ci-centos:https quay.io/pulp/pulp-ci-centos:https
if [ "$VERSION" = "latest-https" ]; then
docker build --build-arg FROM_TAG="https" --file s6_images/pulp_galaxy_ng/Containerfile --tag pulp/pulp-galaxy-ng:https .
docker tag pulp/pulp-galaxy-ng:https ghcr.io/pulp/pulp-galaxy-ng:https
docker tag pulp/pulp-galaxy-ng:https quay.io/pulp/all-in-one-galaxy:https

docker build --build-arg FROM_TAG="https" --file s6_images/pulp/Containerfile --tag pulp/pulp:https .
docker tag pulp/pulp:https ghcr.io/pulp/pulp:https
docker tag pulp/pulp:https quay.io/pulp/all-in-one-pulp:https
else
docker tag pulp/pulp-ci-centos:https pulp/pulp-ci-centos:$VERSION
docker tag pulp/pulp-ci-centos:https ghcr.io/pulp/pulp-ci-centos:$VERSION
docker tag pulp/pulp-ci-centos:https quay.io/pulp/pulp-ci-centos:$VERSION

docker build --build-arg FROM_TAG="https" --file s6_images/pulp/Containerfile --tag pulp/pulp:$VERSION .
docker tag pulp/pulp:$VERSION ghcr.io/pulp/pulp:$VERSION
docker tag pulp/pulp:$VERSION quay.io/pulp/all-in-one-pulp:$VERSION
fi
- name: Test the images
run: |
if [ "$VERSION" = "latest-https" ]; then
s6_images/assets/test.sh "pulp/pulp:https" https
else
s6_images/assets/test.sh "pulp/pulp:$VERSION" https
fi
- name: Github login
env:
PULP_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKER_BOT_USERNAME: ${{ secrets.DOCKER_BOT_USERNAME }}
run: echo "$PULP_GITHUB_TOKEN" | docker login -u "$DOCKER_BOT_USERNAME" --password-stdin ghcr.io
- name: Push ci image to github
run: |
if [ "$VERSION" = "latest-https" ]; then
docker push ghcr.io/pulp/pulp-ci-centos:https
docker push ghcr.io/pulp/pulp-galaxy-ng:https
docker push ghcr.io/pulp/pulp:https
else
docker push ghcr.io/pulp/pulp-ci-centos:$VERSION
docker push ghcr.io/pulp/pulp:$VERSION
docker push ghcr.io/pulp/${{ matrix.image }}:${{ matrix.tag }}
if [ "$IMAGE" = "pulp" ]; then
docker push ghcr.io/pulp/pulp-ci-centos:${{ matrix.tag }}
fi
- name: Docker login
env:
Expand All @@ -188,13 +78,9 @@ jobs:
run: echo "$DOCKER_BOT_PASSWORD" | docker login -u "$DOCKER_BOT_USERNAME" --password-stdin docker.io
- name: Push ci image to dockerhub
run: |
if [ "$VERSION" = "latest-https" ]; then
docker push docker.io/pulp/pulp-ci-centos:https
docker push docker.io/pulp/pulp-galaxy-ng:https
docker push docker.io/pulp/pulp:https
else
docker push docker.io/pulp/pulp:$VERSION
docker push docker.io/pulp/pulp-ci-centos:$VERSION
docker push docker.io/pulp/${{ matrix.image }}:${{ matrix.tag }}
if [ "$IMAGE" = "pulp" ]; then
docker push docker.io/pulp/pulp-ci-centos:${{ matrix.tag }}
fi
- name: Quay login
env:
Expand All @@ -203,13 +89,9 @@ jobs:
run: echo "$QUAY_BOT_PASSWORD" | docker login -u "$QUAY_BOT_USERNAME" --password-stdin quay.io
- name: Push ci image to quay
run: |
if [ "$VERSION" = "latest-https" ]; then
docker push quay.io/pulp/pulp-ci-centos:https
docker push quay.io/pulp/all-in-one-galaxy:https
docker push quay.io/pulp/all-in-one-pulp:https
else
docker push quay.io/pulp/pulp-ci-centos:$VERSION
docker push quay.io/pulp/all-in-one-pulp:$VERSION
docker push "quay.io/pulp/all-in-one-${IMAGE}:${{ matrix.tag }}"
if [ "$IMAGE" = "pulp" ]; then
docker push quay.io/pulp/pulp-ci-centos:${{ matrix.tag }}
fi
- name: Display log on error
if: failure()
Expand Down
Loading
Loading