Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix packaging episode #311

Merged
merged 3 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,7 @@ help:
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

# Live reload site documents for local development
livehtml:
sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
118 changes: 90 additions & 28 deletions content/packaging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,15 +140,17 @@ Sharing packages via PyPI
Once we are able to pip-install the example package locally, we are ready for
upload.

We exercise by uploading to `test-PyPI <https://test.pypi.org/>`__, not the
We exercise by uploading to test-PyPI_, not the
real `PyPI <https://pypi.org/>`__, so that if we mess things up, nothing bad
happens.

We need two more things:

- We will do this using `Twine <https://twine.readthedocs.io/>`__ so you need
to pip install that, too.
- You need an account on `test-PyPI <https://test.pypi.org/>`__.
- You need an account on test-PyPI_

.. _test-PyPI: https://test.pypi.org/

.. highlight:: console

Expand All @@ -165,13 +167,64 @@ And use twine to upload the distribution files to test-PyPI::
$ twine upload -r testpypi dist/*

Uploading distributions to https://test.pypi.org/legacy/
Enter your username:
Enter your password:
Enter your API token:


.. _Create API token: https://test.pypi.org/manage/account/token/

.. note::

To generate an API token, proceed to the `Create API token`_ page in test-PyPI.
You will be prompted for your password.

.. solution:: The long-version for finding the *Create API token* page

1. Log on to test-PyPI_ at https://test.pypi.org
2. In the top-right corner, click on the drop-down menu and click **Account settings** or
follow this `link <https://test.pypi.org/manage/account/#api-tokens>`__.
3. Scroll down to the section **API tokens** and click the button **Add API token**,
which opens up the
`Create API token`_ page.


#. Under **Token name** write something memorable.
It should remind you the *purpose* such as
or the *name of the computer* so you can delete it later.
#. Under **Scope** select ``Entire account (all projects)``.
#. Click on **Create token**.
#. Click on **Copy token** once a long string which starts
with ``pypi-`` is generated.

Once this is done, create yet another virtual environment and try to install from test-PyPI (adapt "myname")::
Paste that token back into terminal where ``twine upload ...`` is running and press ENTER.

$ pip install -i https://test.pypi.org/simple/ calculator-myname
Once this is done, create yet another virtual environment and try to install from test-PyPI (adapt ``myname``).

.. tabs::

.. tab:: Linux / macOS

.. code-block:: console
:emphasize-lines: 4-7

$ python3 -m venv venv-calculator
$ source venv-calculator/bin/activate
$ which python
$ python3 -m pip install \
-i https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
calculator-myname
$ deactivate

.. tab:: Windows

.. code-block:: console
:emphasize-lines: 4

$ python3 -m venv venv-calculator
$ venv-calculator\Scripts\activate
$ where python
$ python3 -m pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ calculator-myname
$ deactivate

Tools that simplify sharing via PyPI
------------------------------------
Expand All @@ -188,6 +241,8 @@ There are at least two tools which try to make the packaging and PyPI interactio
- `Poetry <https://python-poetry.org/>`__
- `Flit <https://flit.pypa.io/>`__

If you upload packages to PyPI or test PyPI often you can create an API token and
`save it in the .pypirc file <https://packaging.python.org/en/latest/specifications/pypirc/#common-configurations>`__.

Building a conda package and share it
-------------------------------------
Expand All @@ -199,42 +254,44 @@ Building a conda package and share it

.. callout:: Prerequisites

To create a conda package, `conda-build` package is required. You may install it with **Anaconda Navigator** or from the command line::
To generate a conda build recipe, the package ``grayskull`` and
to build it, the package ``conda-build`` are required.
You may install these with **Anaconda Navigator** or from the command line::

$ conda install conda-build
$ conda install -n base grayskull conda-build


The simplest way for creating a conda package for your python script is to
first publish it in `PyPI <https://pypi.org/>`__ following the steps explained
above.


Building a python package with conda skeleton pypi
***************************************************
Building a python package with grayskull and conda-build
********************************************************

Once build, the conda package can be installed locally. For this example, we
will use `runtest <https://pypi.org/project/runtest/>`__. `runtest
<https://github.com/bast/runtest>`__ is a numerically tolerant end-to-end test
library for research software.

1. Create pypi skeleton::
1. Generate the *recipe* by executing (``grayskull`` or ``conda grayskull``)::

$ conda skeleton pypi runtest
$ conda grayskull pypi runtest

The command above will create a new folder called `runtest` containing a file `meta.yaml`, the conda recipe for `runtest`.
The command above will create a new folder called `runtest` containing a file `meta.yaml`,
the conda recipe for building the `runtest` package.

2. Edit `meta.yaml` and update requirements:
2. View the contents of `meta.yaml` and ensure requirements :

.. code-block:: yaml

requirements:
host:
- pip
- python
- flit
- flit-core >=2,<4
- pip
run:
- python
- flit

In the requirements above, we specified what is required for the `host <https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#host>`__ and for `running <https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#run>`__ the package.

Expand All @@ -248,7 +305,7 @@ library for research software.

Your package is now ready to be build with conda::

$ conda-build runtest
$ conda build runtest


.. callout:: Conda package location
Expand All @@ -257,11 +314,15 @@ library for research software.

.. code-block:: none

~/anaconda3/conda-bld/win-64/runtest-2.2.1-py38_0.tar.bz2
/home/username/miniforge3/conda-bld/noarch/runtest-2.3.4-py_0.tar.bz2

The prefix `~/anaconda3/` may be different on your machine and depending on your operating system (Linux, Mac-OSX or Windows) the sub-folder `win-64` differs too (for instance `linux-64` on Linux machines).
The prefix ``/home/username/miniforge3/`` may be different on your machine.
depending on your operating system (Linux, Mac-OSX or Windows). The sub-folder is named ``noarch`` since
it is a pure-python package and the recipe indicates the same.

The conda package we have created is specific to your platform (here `win-64`). It can be converted to other platforms using `conda convert <https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs.html#converting-a-package-for-use-on-all-platforms>`__.
If package contained compiled code then the sub-folder would have been named ``win-64`` or ``linux-64``.
It could then be converted to other platforms using
`conda convert <https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs.html#converting-a-package-for-use-on-all-platforms>`__.

4. Check within new environment

Expand All @@ -286,7 +347,10 @@ library for research software.

.. callout:: Building a conda package from scratch

It is possible to build a conda package from scratch without using conda skeleton. We recommend you to check the `conda-build documentation <https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs.html>`__ for more information.
It is possible to build a conda package from scratch without using conda grayskull.
We recommend you to check the
`conda-build documentation <https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs.html>`__
for more information.

To be able to share and install your local conda package anywhere (on other platforms), you would need to upload it to a `conda channel <https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/channels.html>`__ (see below).

Expand All @@ -295,15 +359,13 @@ To be able to share and install your local conda package anywhere (on other plat
Publishing a python package
***************************

- Upload your package to *Anaconda.org*: see instructions `here
<https://docs.conda.io/projects/conda-build/en/latest/user-guide/tutorials/build-pkgs-skeleton.html#optional-uploading-packages-to-anaconda-org>`__.
Please note that you will have to create an account on Anaconda.

- Upload your package to `conda-forge <https://conda-forge.org/>`__:
conda-forge is a conda channel: it contains community-led collection of
recipes, build infrastructure and distributions for the conda package
manager. Anyone can public conda packages to conda-forge if certain
`guidelines <https://conda-forge.org/docs/>`__ are respected.
manager. Anyone can
`publish conda packages to conda-forge <https://conda-forge.org/docs/maintainer/adding_pkgs/>`__
if certain
`guidelines <https://conda-forge.org/docs/maintainer/guidelines/>`__ are respected.

- Upload your package to `bioconda <https://bioconda.github.io/>`_: bioconda is
a very popular channel for the conda package manager specializing in
Expand Down
Loading