Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): Create release workflow #64

Merged
merged 3 commits into from
Dec 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 0 additions & 59 deletions .github/workflows/auto-tag.yml

This file was deleted.

229 changes: 144 additions & 85 deletions .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -1,113 +1,172 @@
# This is a basic workflow to help you get started with uploading to pypi automatically
# https://packaging.python.org/tutorials/packaging-projects/
#
# Before running this workflow in your repository, you will need to set up Secrets in your repository settings:
# - Log in to your (test)PyPI account, go to your account -> your_project -> Publishing
# - Fill in the required fields
# - Create an API token for the repository and this workflow

# - Go to your repository on GitHub, click on the "Settings" tab, and then "Secrets"
# - Add a new secret with the name PYPI_TOKEN and the value is your pypi token
# - Add a new secret with the name PYPI_TEST_TOKEN and the value is your test pypi token
# Then, define the name of your package and the python version you want to use in the env block below.

# This workflow will then automatically build and upload your package to PyPI/TestPypi:
# - When a new commit is pushed to main, it will build and upload to TestPyPI to catch errors early
# - When a new release is created, it will build and upload to the real PyPI
---
env:
PACKAGE_NAME: "fiat_toolbox"
PYTHON_VERSION: "3.10"
name: release

# This workflow will build a Python package and publish it to PyPI when a new tag is created.

# Usage:
# - Create a new tag with the version number in the format v*.*.* where * are integers: `git tag v1.2.3`
# - Push the tag to the repository: `git push --tags`

# Result:
# - Check if the new version is greater than the latest version on PyPI
# - Install dependencies
# - Build the package
# - Publish it to PyPI
# - Create a GitHub release with the tag name and the release notes

# Checklist for using this workflow up for a new project:

# 1. In github settings:
# - Create an environment called `release` (GH_ENV_NAME) and setup the permissions (https://github.com/<ORGANIZATION>/<REPO_NAME>/settings/environments)

# 2. On PyPi:
# - Create the project and add a trusted publisher (https://pypi.org/manage/project/<PACKAGE_NAME>/settings/publishing/ or https://pypi.org/manage/account/publishing if the project is not on pypi yet)
# - Ensure the publisher is configured to use:
# - environment name: `release` (can be changed, but should be the one created in the github settings)
# - the filename of this workflow yml (in this case: publish-to-pypi.yml)

# 3. In this file:
# - In the `env` section (~L45):
# - Update the name of your package (PACKAGE_NAME)
# - Update the version of Python you want to use (PYTHON_VERSION)
# - Update the name name of the environment you created in github settings (GH_ENV_NAME)

# - In the `setup_and_build` job:
# - Update the the shell commands to install your package

name: Build and Upload to PyPI
on:
workflow_run:
workflows: [Update version on main]
types:
- completed
push:
tags:
# https://peps.python.org/pep-0440/
- r"v[0-9]+\.[0-9]+\.[0-9]"

env:
GH_ENV_NAME: "release"
PACKAGE_NAME: "fiat-toolbox"
PYTHON_VERSION: "3.10"

jobs:
build-artifacts: # Install your build env and build your package
details:
runs-on: ubuntu-latest
outputs:
new_version: ${{ steps.release.outputs.new_version }}
tag_name: ${{ steps.release.outputs.tag_name }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: conda-incubator/setup-miniconda@v3
name: Setup Miniconda
with:
auto-update-conda: true
python-version: ${{ env.PYTHON_VERSION }}

- name: Install GDAL
shell: bash -l {0}
run: conda install -c conda-forge gdal
- uses: actions/checkout@v2

- name: Install dependencies and build tools
shell: bash -l {0}
- name: Extract tag and Details
id: release
run: |
python -m pip install --upgrade pip
python -m pip install .
python -m pip install build
if [ "${{ github.ref_type }}" = "tag" ]; then
TAG_NAME=${GITHUB_REF#refs/tags/}
NEW_VERSION=$(echo $TAG_NAME | awk -F'-' '{print $1}')
echo "new_version=$NEW_VERSION" >> "$GITHUB_OUTPUT"
echo "tag_name=$TAG_NAME" >> "$GITHUB_OUTPUT"
echo "Version is $NEW_VERSION"
echo "Tag name is $TAG_NAME"
else
echo "No tag found"
exit 1
fi

- name: Build package
shell: bash -l {0}
run: python -m build
check_pypi:
needs: details
runs-on: ubuntu-latest
steps:
- name: Fetch information from PyPI
run: |
response=$(curl -s https://pypi.org/pypi/${{ env.PACKAGE_NAME }}/json || echo "{}")
latest_previous_version=$(echo $response | jq --raw-output "select(.releases != null) | .releases | keys_unsorted | last")
if [ -z "$latest_previous_version" ]; then
echo "Package not found on PyPI."
latest_previous_version="0.0.0"
fi
echo "Latest version on PyPI: $latest_previous_version"
echo "latest_previous_version=$latest_previous_version" >> $GITHUB_ENV

- uses: actions/upload-artifact@v4
with:
name: releases
path: dist
- name: Compare versions and exit if not newer
run: |
NEW_VERSION=${{ needs.details.outputs.new_version }}
LATEST_VERSION=$latest_previous_version
if [ "$(printf '%s\n' "$LATEST_VERSION" "$NEW_VERSION" | sort -rV | head -n 1)" != "$NEW_VERSION" ] || [ "$NEW_VERSION" == "$LATEST_VERSION" ]; then
echo "The new version $NEW_VERSION is not greater than the latest version $LATEST_VERSION on PyPI."
exit 1
else
echo "The new version $NEW_VERSION is greater than the latest version $LATEST_VERSION on PyPI."
fi

test-built-dist: # Install the built package and test it has been built correctly
needs: build-artifacts
setup_and_build:
needs: [details, check_pypi]
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
steps:
- uses: actions/checkout@v4

- uses: actions/download-artifact@v4
with:
name: releases
path: dist
- name: List contents of built dist
run: |
ls -ltrh
ls -ltrh dist

- uses: conda-incubator/setup-miniconda@v3
name: Setup Miniconda
with:
auto-update-conda: true
activate-environment: publish
python-version: ${{ env.PYTHON_VERSION }}

- name: Install GDAL
shell: bash -l {0}
run: conda install -c conda-forge gdal
channels: conda-forge

- name: Install Build tools
shell: bash -el {0}
run: |
conda run -n publish pip install --upgrade build pip setuptools

- name: Verify the built dist/wheel is valid
- name: Install dependencies
shell: bash -el {0}
run: |
python -m pip install dist/*.whl
python -c "import ${{ env.PACKAGE_NAME }}; print(f'{${{ env.PACKAGE_NAME }}}-{${{ env.PACKAGE_NAME }}.__version__}')"

upload-to-pypi:
environment: release
conda install -n publish gdal --channel conda-forge
conda run -n publish pip install .

- name: Build source and wheel distribution
run: |
conda run -n publish python -m build

- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: dist
path: dist/

pypi_publish:
name: Upload release to PyPI
needs: [setup_and_build, details]
runs-on: ubuntu-latest
environment:
name: ${{ env.GH_ENV_NAME }}
permissions:
id-token: write
needs: test-built-dist
steps:
- name: Download artifacts
uses: actions/download-artifact@v3
with:
name: dist
path: dist/

- name: Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

github_release:
name: Create GitHub Release
needs: [setup_and_build, details]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/download-artifact@v4
- name: Checkout Code
uses: actions/checkout@v3
with:
name: releases
path: dist
- name: Publish package to PyPI
uses: pypa/[email protected]
fetch-depth: 0

- name: Download artifacts
uses: actions/download-artifact@v3
with:
user: __token__
password: ${{ secrets.PYPI_TOKEN }}
verbose: true
name: dist
path: dist/

- name: Create GitHub Release
id: create_release
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release create ${{ needs.details.outputs.tag_name }} dist/* --title ${{ needs.details.outputs.tag_name }} --generate-notes
Loading