PyOCI

Publish and download (private) python packages using an OCI registry for storage.

Why PyOCI

To not have to rely on yet-another-cloud-provider for private Python packages, PyOCI, makes ghcr.io act like a python index.
In addition, this completely removes the need for separate access management as GitHub Packages access control applies.

Most subscriptions with cloud providers include an OCI (docker image) registry where private containers can be published and distributed from.

PyOCI allows using any (private) OCI registry as a python package index, as long as it implements the OCI distribution specification. It acts as a proxy between pip and the OCI registry.

An instance of PyOCI is available at https://pyoci.com, to use this proxy, please see the Getting started.

Tested registries:

• ghcr.io
• Azure Container Registry

Published packages will show up in the OCI registry UI:

Getting started

To install a package with pip using PyOCI:

pip install --index-url="http://<username>:<password>@<pyoci-url>/<OCI-registry-url>/<namespace>/" <package-name>

• <pyoci-url>: https://pyoci.com
• <OCI-registry-url>: URL of the OCI registry to use.
• <namespace>: namespace within the registry, for most registries this is the username or organization name.

Example installing package hello-world from organization allexveldman using ghcr.io as the registry:

pip install --index-url="https://$GITHUB_USER:[email protected]/ghcr.io/allexveldman/" hello-world

Warning

If the package contains dependencies from regular pypi, these will not resolve.

Pip does not have a proper way of indicating you only want to resolve <package-name> through PyOCI and it's dependencies through pypi. Poetry does provide you with a way to do this.

For more examples, including how to publish a package, see the examples.

Host your own

If you don't want, or can't, use https://pyoci.com, you can host your own using the docker container.

docker run ghcr.io/allexveldman/pyoci:latest

Note that only HTTP is support at this moment, PyOCI is expected to run behind a reverse proxy that handles TLS termination, or a trusted environment.

Environment variables

• PORT: port to listen on, defaults to 8080.
• PYOCI_PATH: Host PyOCI on a subpath, for example: PYOCI_PATH="/acme-corp".
• PYOCI_MAX_BODY: Limit the maximum accepted body size in bytes when publishing packages, defaults to 50MB.
• OTLP_ENDPOINT: If set, forward logs, traces, and metrics to this OTLP collector endpoint every 30s.
• OTLP_AUTH: Full Authorization header value to use when sending OTLP requests.
• RUST_LOG: Log filter, defaults to info.

The following environment variables will be added as attributes to the OTLP resources:

• DEPLOYMENT_ENVIRONMENT -> deployment.environment

Set by Azure Container App, can change if I every decide to move host:

• CONTAINER_APP_NAME -> k8s.container.name
• CONTAINER_APP_REVISION -> k8s.pod.name
• CONTAINER_APP_REPLICA_NAME -> k8s.replicaset.name

Add Labels to your package

Labels can be added to your package by including them as a PyOCI :: Label :: <Key> :: <Value> classifier of the package. If the classifiers are found in the package upload request, the key-value pairs will be added as annotations (aka labels in docker terms) to the OCI image.

Note that these classifiers are case-sensitive and non-standard.

Authentication

Pip's Basic authentication is forwarded as-is to the target registry as part of the token authentication flow.

Changing a package

PyOCI will refuse to upload a package file if the package name, version and architecture already exist. To update an existing file, delete it first and re-publish it.

Deleting a package

There is no formal specification for deleting python packages, instead you can use the OCI registry provided methods to delete your package.

PyOCI also supports deleting a package file using DELETE /<registry>/<namespace>/<package-name>/<filename>, support depends on the underlying registry's support for the content management section of the OCI Distribution specification.

Renovate + ghcr.io

As PyOCI acts as a private pypi index, Renovate needs to be configured to use credentials for your private packages. (https://docs.renovatebot.com/getting-started/private-packages/) To prevent having to check-in encrypted secrets you can:

1. Self-host renovate as a github workflow
2. Set package: read permissions for the workflow
3. Pass the GITHUB_TOKEN as an environment variable to Renovate
4. Add a hostRule for the Renovate runner to apply basic auth for pyoci using the environment variable
5. In the package settings of the private package give the repository running renovate read access.

Note that at the time of writing, GitHub App Tokens can't be granted read:package permissions, this is why you'll need to use the GITHUB_TOKEN.

.github/workflows/renovate.yaml

...
concurrency:
  group: Renovate

# Allow the GITHUB_TOKEN to read packages
permissions:
  contents: read
  packages: read

jobs:
  renovate:
    ...
      - name: Self-hosted Renovate
        uses: renovatebot/[email protected]
        with:
          configurationFile: config.js
          token: '${{ steps.get_token.outputs.token }}'
        env:
          RENOVATE_PYOCI_USER: pyocibot
          RENOVATE_PYOCI_TOKEN: ${{ secrets.GITHUB_TOKEN }}

config.js

module.exports = {
  ...
  hostRules: [
    {
      matchHost: "pyoci.com",
      hostType: "pypi",
      username: process.env.RENOVATE_PYOCI_USER,
      password: process.env.RENOVATE_PYOCI_TOKEN
    },
  ],
};

Contributing

See the contributing