From 79a884f90e0bff816805c62d52bbf0eff416bbdd Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 20 Nov 2024 17:29:17 +0200 Subject: [PATCH 01/18] fork python to a directory --- python.rst | 10 ++++++++++ python.md => python/overview.md | 11 +++++++++++ 2 files changed, 21 insertions(+) create mode 100644 python.rst rename python.md => python/overview.md (96%) diff --git a/python.rst b/python.rst new file mode 100644 index 0000000..b3a6b48 --- /dev/null +++ b/python.rst @@ -0,0 +1,10 @@ +Your Python Environment +======================= + +This section discusses setting up the Python side of your development environment, +including choosing a tool to manage your environment and working with the different tools. + +.. toctree:: + + python/overview + diff --git a/python.md b/python/overview.md similarity index 96% rename from python.md rename to python/overview.md index 3c272fd..a46b753 100644 --- a/python.md +++ b/python/overview.md @@ -43,6 +43,17 @@ If you don't see 3.11 as an option, you may need to install it first. Follow these steps if you want to manage your virtual environments outside your IDE. Using `venv` is recommended if you're not sure which option to use. +#### Using uv (recommended) + +It's recommended that new projects use uv to manage their Python environments. +It is faster and simpler to use. + +To use uv... + +FILL THIS IN. + +**Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** + #### Using venv The easiest way to set up a virtual environment manually is to use Python's built in From 9ab597bba6eee28889d5c0bb87ae9409ecb0fe69 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 20 Nov 2024 17:44:27 +0200 Subject: [PATCH 02/18] rename + split files --- python/packages.md | 109 ++++++++++++++++++++++++ python/{overview.md => setup.md} | 138 ++++--------------------------- 2 files changed, 124 insertions(+), 123 deletions(-) create mode 100644 python/packages.md rename python/{overview.md => setup.md} (53%) diff --git a/python/packages.md b/python/packages.md new file mode 100644 index 0000000..258fd19 --- /dev/null +++ b/python/packages.md @@ -0,0 +1,109 @@ +Working with Python Packages (pip-tools) +======================================== + +Pegasus uses [pip tools](https://github.com/jazzband/pip-tools) to manage Python dependencies. +This allows for more explicit dependency management than a standard `requirements.txt` file. + +### Requirements Files + +Pegasus has multiple requirements files, which live in the `requirements/` folder. +For each set of requirements there are two files, one ending in `.in` and the other ending in `.txt`. + +The files ending in `requirements.in` have the first-class packages your app depends on. +They do not have versions in them, though you can add version numbers if you want to. +**These are the files that you should edit when adding/removing packages**. + +The files ending in `requirements.txt` have the full list of packages your app depends on, +including the dependencies of your dependencies (recursively). +This file is automatically generated from the `.in` counterpart, and **should typically not be edited by hand**. + +The `requirements.in`/`.txt` files are the main requirements for your application, `dev-requirements.in`/`.txt` files +are requirements for development-only, and `prod-requirements.in`/`.txt` are for production-only. + +### Working with requirements + +To modify the requirements files, you first need to install `pip-tools`. +It is included as a dependency in the `dev-requirements.txt` file so if you've followed the local setup +steps it should already be installed. + +Then follow the instructions below, depending on what you want to do: + +#### Adding or removing a package + +To add a package, add the package name to `requirements/requirements.in`. +To remove a package, remove it from `requirements/requirements.in`. + +After finishing your edits, rebuild your `requirements.txt` file by running: + +``` +# native version +pip-compile requirements/requirements.in + +# docker version +make pip-compile +``` + +After running this you should see the package and its dependencies added to the `requirements.txt` file. + +From there you can install the new dependencies, as [described below](#installing-packages). + +#### Upgrading a package + +To upgrade a package, you can run the following command. In this example we are upgrading `django`: + +``` +# native version +pip-compile --upgrade-package django requirements/requirements.in + +# docker version +make pip-complie ARGS="--upgrade-package django" +``` + +To upgrade *all* packages, you can run: + +``` +# native version +pip-compile --upgrade requirements/requirements.in + +# docker version +make pip-compile ARGS="--upgrade" +``` + +From there you can install the new dependencies, as [described below](#installing-packages). + +#### Installing Packages + +If you're running Python natively, you can install your packages with the following command. +Run this after activating your virtual environment: + +``` +pip install -r requirements/requirements.txt +``` + +In Docker your Python packages are installed at container *build* time. +This means that any time you want to change your installed new packages, you have to rebuild your container. + +You can do this by running + +``` +docker compose build +``` + +Confusingly, running `pip install` or `docker compose exec web pip install` does *not* work. + +#### The `make requirements` shortcut for Docker + +Pegasus ships with a convenience target for rebuilding requirements with Docker. +Any time you make changes to a `requirements.in` file you can run it with: + +``` +make requirements +``` + +Behind the scenes this will: + +1. Rebuild all your `-requirements.txt` files from your `-requirements.in` files with `uv`. +2. Rebuild your containers (installing the new packages). +3. Restart your containers. + +For more information, see the [docker documentation](docker.md#updating-python-packages). diff --git a/python/overview.md b/python/setup.md similarity index 53% rename from python/overview.md rename to python/setup.md index a46b753..d806a27 100644 --- a/python/overview.md +++ b/python/setup.md @@ -1,7 +1,7 @@ -Your Python Environment -======================= +Environment Setup +================= -## Choosing between Docker and Native Python +## Choosing between Docker and native Python You can either run Python in a Docker container or natively on your development machine. @@ -26,6 +26,18 @@ which avoids dependency conflicts and makes it easier to run multiple Python app Follow one of the sections below depending on how you want to manage your virtualenvs. +### Using uv (recommended) + +It's recommended that new projects use uv to manage their Python environments. +It is faster and simpler to use than other alternatives, and can even install +and set up Python for you. + +To use uv... + +FILL THIS IN. + +**Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** + ### Using your IDE Many IDEs will manage your environments for you. @@ -43,17 +55,6 @@ If you don't see 3.11 as an option, you may need to install it first. Follow these steps if you want to manage your virtual environments outside your IDE. Using `venv` is recommended if you're not sure which option to use. -#### Using uv (recommended) - -It's recommended that new projects use uv to manage their Python environments. -It is faster and simpler to use. - -To use uv... - -FILL THIS IN. - -**Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** - #### Using venv The easiest way to set up a virtual environment manually is to use Python's built in @@ -121,112 +122,3 @@ workon {{ project_name }} You can use `virtualenvwrapper` no matter how you created the environment. It provides a nice set of helper tools, but can be a bit finicky to set up. - -## Working with Python Packages - -Pegasus uses [pip tools](https://github.com/jazzband/pip-tools) to manage Python dependencies. -This allows for more explicit dependency management than a standard `requirements.txt` file. - -### Requirements Files - -Pegasus has multiple requirements files, which live in the `requirements/` folder. -For each set of requirements there are two files, one ending in `.in` and the other ending in `.txt`. - -The files ending in `requirements.in` have the first-class packages your app depends on. -They do not have versions in them, though you can add version numbers if you want to. -**These are the files that you should edit when adding/removing packages**. - -The files ending in `requirements.txt` have the full list of packages your app depends on, -including the dependencies of your dependencies (recursively). -This file is automatically generated from the `.in` counterpart, and **should typically not be edited by hand**. - -The `requirements.in`/`.txt` files are the main requirements for your application, `dev-requirements.in`/`.txt` files -are requirements for development-only, and `prod-requirements.in`/`.txt` are for production-only. - -### Working with requirements - -To modify the requirements files, you first need to install `pip-tools`. -It is included as a dependency in the `dev-requirements.txt` file so if you've followed the local setup -steps it should already be installed. - -Then follow the instructions below, depending on what you want to do: - -#### Adding or removing a package - -To add a package, add the package name to `requirements/requirements.in`. -To remove a package, remove it from `requirements/requirements.in`. - -After finishing your edits, rebuild your `requirements.txt` file by running: - -``` -# native version -pip-compile requirements/requirements.in - -# docker version -make pip-compile -``` - -After running this you should see the package and its dependencies added to the `requirements.txt` file. - -From there you can install the new dependencies, as [described below](#installing-packages). - -#### Upgrading a package - -To upgrade a package, you can run the following command. In this example we are upgrading `django`: - -``` -# native version -pip-compile --upgrade-package django requirements/requirements.in - -# docker version -make pip-complie ARGS="--upgrade-package django" -``` - -To upgrade *all* packages, you can run: - -``` -# native version -pip-compile --upgrade requirements/requirements.in - -# docker version -make pip-compile ARGS="--upgrade" -``` - -From there you can install the new dependencies, as [described below](#installing-packages). - -#### Installing Packages - -If you're running Python natively, you can install your packages with the following command. -Run this after activating your virtual environment: - -``` -pip install -r requirements/requirements.txt -``` - -In Docker your Python packages are installed at container *build* time. -This means that any time you want to change your installed new packages, you have to rebuild your container. - -You can do this by running - -``` -docker compose build -``` - -Confusingly, running `pip install` or `docker compose exec web pip install` does *not* work. - -#### The `make requirements` shortcut for Docker - -Pegasus ships with a convenience target for rebuilding requirements with Docker. -Any time you make changes to a `requirements.in` file you can run it with: - -``` -make requirements -``` - -Behind the scenes this will: - -1. Rebuild all your `-requirements.txt` files from your `-requirements.in` files with `uv`. -2. Rebuild your containers (installing the new packages). -3. Restart your containers. - -For more information, see the [docker documentation](docker.md#updating-python-packages). From 8551fd445aa35d5368d31ef5cf31de20c7834ee9 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 20 Nov 2024 17:44:34 +0200 Subject: [PATCH 03/18] add uv --- python.rst | 4 +++- python/uv.md | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 1 deletion(-) create mode 100644 python/uv.md diff --git a/python.rst b/python.rst index b3a6b48..23e569d 100644 --- a/python.rst +++ b/python.rst @@ -6,5 +6,7 @@ including choosing a tool to manage your environment and working with the differ .. toctree:: - python/overview + python/setup + python/packages + python/uv diff --git a/python/uv.md b/python/uv.md new file mode 100644 index 0000000..bacc6a9 --- /dev/null +++ b/python/uv.md @@ -0,0 +1,37 @@ +Working with Python Packages (uv) +================================= + +Recent versions of Pegasus use [uv](https://docs.astral.sh/uv/) to manage Python packages. +It provides all the functionality of pip-tools while being much faster and offering more flexibility +and features. + +### Requirements Files + +`uv` uses two files to manage requirements. The first is a `pyproject.toml` file, which contains the +base list of packages. The `pyproject.toml` file also supports dependency groups, which are used +for development and production requirements. +`pyproject.toml` replaces the previous `requirements.in`, `dev-requirements.in`, and `prod-requirements.in` files. + +The second file is the `uv.lock` file. This file contains the pinned versions of dependencies +that are used by the project's environment. +This file is automatically generated from the `pyproject.toml` file and *should not be edited by hand*. +`uv.lock` replaces the previous `requirements.txt`, `dev-requirements.txt`, and `prod-requirements.txt` files. + +#### Adding or removing a package + +TODO + +#### Upgrading a package + +TODO + +#### Installing Packages + +TODO + +#### The `make requirements` shortcut for Docker + +TODO + + + From 0c1d4aced65e45285eb36cddccaf117a9a4e6c61 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 09:47:18 +0200 Subject: [PATCH 04/18] add bash markup --- python/packages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/packages.md b/python/packages.md index 258fd19..13d4289 100644 --- a/python/packages.md +++ b/python/packages.md @@ -51,7 +51,7 @@ From there you can install the new dependencies, as [described below](#installin To upgrade a package, you can run the following command. In this example we are upgrading `django`: -``` +```bash # native version pip-compile --upgrade-package django requirements/requirements.in From a02191e9213772022334bdf119894b9ddd5eb12c Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 09:47:20 +0200 Subject: [PATCH 05/18] uv docs --- python/uv.md | 50 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/python/uv.md b/python/uv.md index bacc6a9..9093d42 100644 --- a/python/uv.md +++ b/python/uv.md @@ -19,19 +19,55 @@ This file is automatically generated from the `pyproject.toml` file and *should #### Adding or removing a package -TODO +To add or remove packages you can run the following commandss: -#### Upgrading a package +```bash +# native version +uv add +uv remove + +# docker version +make uv add +make uv remove +``` + +If you're using natively this is all you have to do! The command will update your `pyproject.toml` file, +your `uv.lock` file, and sync your virtual environment. + +On Docker, you will have to also rebuild the container. You can do that with: -TODO +```bash +make build +make restart +``` + +The `make requiements` command can also be used to sync your `uv.lock` file and rebuild / restart your containers. + +#### Upgrading a package -#### Installing Packages +You can upgrade a package with: -TODO +```bash +# native version - update the lockfile +uv lock --upgrade-package +# native version - update the lockfile and sync the virtual environment +uv sync --upgrade-package -#### The `make requirements` shortcut for Docker +# docker version +make uv "lock --upgrade-package wagtail" +``` -TODO +You can upgrade *all* packages with: +```bash +# native version - update the lockfile +uv lock --upgrade +# native version - update the lockfile and sync the virtual environment +uv sync --upgrade +# docker version +make uv "lock --upgrade" +``` +Like with adding packages, you'll have to rebuild and restart Docker containers for the updated +environment to work. From 0490891f5c372d8e360bc50b102b1f7e279895c0 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 09:56:07 +0200 Subject: [PATCH 06/18] add uv setup docs --- python/setup.md | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/python/setup.md b/python/setup.md index d806a27..3501704 100644 --- a/python/setup.md +++ b/python/setup.md @@ -28,13 +28,27 @@ Follow one of the sections below depending on how you want to manage your virtua ### Using uv (recommended) -It's recommended that new projects use uv to manage their Python environments. +It's recommended that new projects use [uv](https://docs.astral.sh/uv/) to manage their Python environments. It is faster and simpler to use than other alternatives, and can even install and set up Python for you. -To use uv... +To use uv first [install it](https://docs.astral.sh/uv/getting-started/installation/). -FILL THIS IN. +Then in your project root directory run: + +```bash +uv sync +``` + +After that you should be good to go! +You should be able to use `uv run` to run any Python command on your project, +or you can run: + +```bash +source .venv/bin/activate +``` + +To use Python and other commands normally. **Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** From ac1ae70adfbf26dc9055fb2974019f5268882173 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 10:02:16 +0200 Subject: [PATCH 07/18] tweak --- python/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/setup.md b/python/setup.md index 3501704..c4e094a 100644 --- a/python/setup.md +++ b/python/setup.md @@ -48,7 +48,7 @@ or you can run: source .venv/bin/activate ``` -To use Python and other commands normally. +in your project root to use Python and other commands normally. **Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** From d6e5049595f7b7d63d2f2c6d5627b3271f32f696 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 11:44:18 +0200 Subject: [PATCH 08/18] uv blurb --- release-notes.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/release-notes.md b/release-notes.md index aca8f6d..de38488 100644 --- a/release-notes.md +++ b/release-notes.md @@ -3,6 +3,18 @@ Version History and Release Notes Releases of [SaaS Pegasus: The Django SaaS Boilerplate](https://www.saaspegasus.com/) are documented here. +### UV support! + +This release adds full support for [uv](https://docs.astral.sh/uv/) as a replacement package manager. +You can use uv by selecting the new "use uv" option in the project configuration page. + +When you select uv the following changes will be made: + +- All requirements files are removed. +- Main project requirements will be listed in your `pyproject.toml` file. +- Pinned project requirements will be listed in a new `uv.lock` file. +- Docker containers (in development and production) will use `uv` to set up and manage the Python environment. + ## Version 2024.11.3 From 20e5fe8b77d75a5b4ca8baac5c4902ea91664427 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 21 Nov 2024 16:03:27 +0200 Subject: [PATCH 09/18] add migrating doc --- cookbooks.md | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/cookbooks.md b/cookbooks.md index 0a25f7e..398ce4c 100644 --- a/cookbooks.md +++ b/cookbooks.md @@ -24,6 +24,100 @@ python ./manage.py promote_user_to_superuser yourname@example.com Now you should be able to access the django admin at http://localhost:8000/admin +## Migrating from pip-tools to uv + +To migrate your project from pip-tools to uv follow these steps. + +### Install uv + +If you haven't already, [install uv](https://docs.astral.sh/uv/getting-started/installation/): + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +### Update your project code + +It's recommended to do this in two steps: + +1. [Upgrade your project](./upgrading.md) to the latest Pegasus version, keeping your pacakge manager as "pip-tools". + Merge all conflicts and ensure your project is working properly on this version. +2. Then, change the package manager from pip-tools to uv in your project settings and do another upgrade/pull request. + +At this point you will likely have conflicts in your requirements files, but hopefully nowhere else. +See the next sections for resolving these. + +### Prepare to resolve conflicts + +First, follow the github instructions to merge your project on your local machine, by checking out the pegasus upgrade +branch and merging the main branch into it. +You will have to update the command below with the exact branch name of the pull request created by Pegasus: + +``` +git fetch origin +git checkout pegasus-- +git merge main +``` + +At this point you'll have a partially merged branch with conflicts. + +### Migrate your requirements.in files + +The uv build of Pegasus no longer uses requirements files, so any changes you've made to these will need +to be migrated to `pyproject.toml` and `uv.lock`. + +You can use the [reqs-sync](https://github.com/saaspegasus/reqs-sync/) package to help with this. +Follow the steps below for any file with conflicts. + +To migrate your main *requirements.in* file: + +```bash +uv tool run reqs-sync reqs-to-toml requirements/requirements.in +``` + +To migrate your development *dev-requirements.in* file: + +```bash +uv tool run reqs-sync reqs-to-toml requirements/dev-requirements.in --group=dev +``` + +To migrate your production *prod-requirements.in* file: + +```bash +uv tool run reqs-sync reqs-to-toml requirements/prod-requirements.in --group=prod +``` + +These commands should copy all project requirements from your `requirements.in` file(s) to your `pyproject.toml` file +(into the appropriate group, if necessary). + +### Update your uv.lock file + +Next you should rebuild your `uv.lock` file from the updated `pyproject.toml` file: + +```bash +uv lock +``` + +You should then check the versions that were added to the `uv.lock` file and update any as +needed based on the versions your requirements.txt files. + +### Test the migration + +Run your project (`uv run python manage.py runserver`) and verify everything works as expected. + +### Remove your requirements files + +Finally, run: + +```bash +git rm requirements/*` +``` + +To remove all your requirements files. + +Congratulations, you've migrated to uv! +Resolve any other conflicts, push and merge your code, and you're done! + ## Migrating to auto-formatted code As of February, 2023 all Pegasus projects have the option to auto-format your Python code. From 7b99558b5d6f6eae5a2628c59c772486054677ec Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 27 Nov 2024 14:54:05 +0200 Subject: [PATCH 10/18] uv-ify getting started docs --- getting-started.md | 33 ++++++++++----------- python/setup.md | 73 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 72 insertions(+), 34 deletions(-) diff --git a/getting-started.md b/getting-started.md index 141461b..7a896ba 100644 --- a/getting-started.md +++ b/getting-started.md @@ -67,28 +67,13 @@ Then skip ahead to the [post-install steps](getting-started.md#post-installation ## Get up and running with native Python -Follow these instructions to run your application in your system's Python. If you're using Docker you can skip this section. -*A video walkthrough of most of these steps is [available here](https://www.youtube.com/watch?v=kUoKm81OFqk).* +### Set up your Python environment -### Install Prerequisites +There are several ways of setting up your Python environment. -If you haven't already, first install Python version 3.11. - -On Mac and windows you can [download Python 3.11 installers from here](https://www.python.org/downloads/). -On Ubuntu it's recommended to [use the deadsnakes repo](https://www.debugpoint.com/install-python-3-11-ubuntu/). - -*Note: running on older Python versions may work, but 3.11 is what's tested and supported.* - -If you're using Postgres, you'll also want to make sure [you have it installed](https://www.postgresql.org/download/). -Depending on how you install it, you may also need to create a user account that can create and manage databases. - -To use [celery](./celery.md) you will also need to [install Redis](https://redis.io/docs/getting-started/installation/). - -### Setup a Python 3.11 virtual environment - -See [the docs on Virtual Environments](/python/#using-native-system-python-with-virtual-environments) for details on this process. +See [this page](./python/setup.md) for information on choosing an option and setting up your environment. ### Enter the project directory @@ -100,6 +85,9 @@ You should see a lot of newly created files for your project including a `manage ### Install package requirements +If you used `uv` your packages should already be installed. +If you chose a different option, install them now by running: + ```shell pip install -r dev-requirements.txt # for production installs use @@ -149,18 +137,27 @@ Followed by the password for the postgres user. ### Create database migrations ```bash +# with uv +uv run manage.py makemigrations +# or with normal venv python ./manage.py makemigrations ``` ### Run database migrations ```bash +# with uv +uv run manage.py migrate +# or with normal venv python ./manage.py migrate ``` ### Run server ```bash +# with uv +uv run manage.py runserver +# or with normal venv python ./manage.py runserver ``` diff --git a/python/setup.md b/python/setup.md index c4e094a..6a2dcdf 100644 --- a/python/setup.md +++ b/python/setup.md @@ -32,17 +32,51 @@ It's recommended that new projects use [uv](https://docs.astral.sh/uv/) to manag It is faster and simpler to use than other alternatives, and can even install and set up Python for you. -To use uv first [install it](https://docs.astral.sh/uv/getting-started/installation/). +*Note: uv support is only available from Pegasus version 2024.12 onwards.* -Then in your project root directory run: +To set up Python with uv, first [install uv](https://docs.astral.sh/uv/getting-started/installation/): + +On Linux / Mac: + +```bash +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +On Windows: + +```bash +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" +``` + +After installing `uv`, go into your project directory and run: ```bash uv sync ``` -After that you should be good to go! -You should be able to use `uv run` to run any Python command on your project, -or you can run: +This should: + +1. Install the right version of Python (if necessary). +2. Create a new virtual environment in a `.venv` folder inside your project. +3. Install all the project dependencies. + +To see if it worked, run: + +```bash +uv run manage.py shell +``` + +If you get a Python shell that looks something like this, it worked! + +``` +$ uv run manage.py shell +Python 3.12.6 (main, Sep 9 2024, 22:11:19) [Clang 18.1.8 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +(InteractiveConsole) +>>> +``` + +You should be able to use `uv run` to run any Python command on your project, or you can run: ```bash source .venv/bin/activate @@ -50,9 +84,20 @@ source .venv/bin/activate in your project root to use Python and other commands normally. -**Note: uv is currently in beta testing but is planned to become the default set up for new projects soon.** +### Using another option -### Using your IDE +There are many other options for setting up your environment. +Unlike `uv`, most of these require having Python installed on your machine, +so if you haven't already, first install Python version 3.11+: + +- On Mac and windows you can [download Python 3.11 installers from here](https://www.python.org/downloads/). +- On Ubuntu it's recommended to [use the deadsnakes repo](https://www.debugpoint.com/install-python-3-11-ubuntu/). + +*Note: running on older Python versions may work, but 3.11 is what's tested and supported.* + +After installing Python, set up your virtual environment through one of the following methods: + +#### Using your IDE Many IDEs will manage your environments for you. This is a great and simple option that you won't have to fiddle with. @@ -64,11 +109,6 @@ Check your specific IDE's docs for guidance on how to do this. **Be sure to choose Python 3.11 when setting up your virtual environment.** If you don't see 3.11 as an option, you may need to install it first. -### Manually managing environments - -Follow these steps if you want to manage your virtual environments outside your IDE. -Using `venv` is recommended if you're not sure which option to use. - #### Using venv The easiest way to set up a virtual environment manually is to use Python's built in @@ -78,9 +118,10 @@ The easiest way to set up a virtual environment manually is to use Python's buil python3.11 -m venv /path/to/environment ``` -In the command below, you should replace `python3.11` with the Python version you are using (3.9 or higher), and +In the command below, you should replace `python3.11` with the Python version you are using, and `/path/to/environment/` with the location on your system where you want to store the environment. -This location can be somewhere in your project directory or anywhere else on your system. +This location can be somewhere in your project directory (`.venv` and `venv` are common choices) +or anywhere else on your system. `/home//.virtualenvs/` is a common choice that works well with `virtualenvwrapper` (see below). To activate/use the environment run: @@ -117,7 +158,7 @@ And, like `venv`, **you will need to activate this environment every time you wo #### Using virtualenvwrapper [Virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/) is an optional convenience -tool that helps manage virtural environments. +tool that helps manage virtual environments. You can use it with either `venv` or `virtualenv` above. If you choose to use `virtualenvwrapper` you can use the following command to create your environment. @@ -134,5 +175,5 @@ Then to activate the environment you use: workon {{ project_name }} ``` -You can use `virtualenvwrapper` no matter how you created the environment. +Note: You can use `virtualenvwrapper` no matter how you created the environment. It provides a nice set of helper tools, but can be a bit finicky to set up. From bf314bf2850fbd305ec4aaa16537a7fc2dacb93a Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 27 Nov 2024 15:11:20 +0200 Subject: [PATCH 11/18] overhaul python setup a bit --- python/setup.md | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/python/setup.md b/python/setup.md index 6a2dcdf..921e955 100644 --- a/python/setup.md +++ b/python/setup.md @@ -3,7 +3,8 @@ Environment Setup ## Choosing between Docker and native Python -You can either run Python in a Docker container or natively on your development machine. +You can either run Python in a Docker container or natively on your development machine, +through various different options. Docker is easier to set up and provides a more consistent way to package your application, however it is slower, takes more resources, and is more complex to integrate with IDEs, debuggers, @@ -12,27 +13,21 @@ and other development tools. Native Python can be more difficult to set up, especially on Windows, but once it is working it is typically easier to work with. -If you're not sure which you want, it's recommended to start with Docker and switch to native Python -if you are unhappy with the Docker experience. +Docker and uv are the recommended options. +You can pick either of these and then switch if you run into problems. ## Using Docker See the [Docker documentation](/docker.md) to set up your development environment with Docker. -## Using Native / System Python (with Virtual Environments) - -If you're not using Docker, it's strongly recommended that you set up your project in a virtual environment, -which avoids dependency conflicts and makes it easier to run multiple Python apps on your machine. - -Follow one of the sections below depending on how you want to manage your virtualenvs. +## Using uv -### Using uv (recommended) - -It's recommended that new projects use [uv](https://docs.astral.sh/uv/) to manage their Python environments. +It's recommended that new projects not using docker use [uv](https://docs.astral.sh/uv/) to manage their Python environments. It is faster and simpler to use than other alternatives, and can even install and set up Python for you. -*Note: uv support is only available from Pegasus version 2024.12 onwards.* +***Note: uv support is only available from Pegasus version 2024.12 onwards. +To use uv you must select it under the "Python package manager" setting in your project configuration.*** To set up Python with uv, first [install uv](https://docs.astral.sh/uv/getting-started/installation/): @@ -84,10 +79,12 @@ source .venv/bin/activate in your project root to use Python and other commands normally. -### Using another option +## Using Native / System Python (with Virtual Environments) + +The following are other options---which are typically recommended for developers +who are already familiar with Python and one of these choices. -There are many other options for setting up your environment. -Unlike `uv`, most of these require having Python installed on your machine, +Unlike `docker` and `uv`, most of these require having Python installed on your machine, so if you haven't already, first install Python version 3.11+: - On Mac and windows you can [download Python 3.11 installers from here](https://www.python.org/downloads/). From 7a1ec217ff425586e01cf85ecbd8b303b5b9e147 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 27 Nov 2024 15:24:00 +0200 Subject: [PATCH 12/18] update docs --- python/uv.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/python/uv.md b/python/uv.md index 9093d42..7103a0c 100644 --- a/python/uv.md +++ b/python/uv.md @@ -2,7 +2,7 @@ Working with Python Packages (uv) ================================= Recent versions of Pegasus use [uv](https://docs.astral.sh/uv/) to manage Python packages. -It provides all the functionality of pip-tools while being much faster and offering more flexibility +It provides all the functionality of `pip-tools` while being much faster and offering more flexibility and features. ### Requirements Files @@ -69,5 +69,10 @@ uv sync --upgrade make uv "lock --upgrade" ``` -Like with adding packages, you'll have to rebuild and restart Docker containers for the updated -environment to work. +Like with adding packages, if you're using Docker, you'll have to rebuild and restart Docker containers for the updated +environment to work: + +```bash +make build +make restart +``` From 9e0b59c06b759c147c3e82ba19c948d2a1f958e5 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Wed, 27 Nov 2024 15:24:03 +0200 Subject: [PATCH 13/18] update uv release notes --- release-notes.md | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/release-notes.md b/release-notes.md index de38488..befa13f 100644 --- a/release-notes.md +++ b/release-notes.md @@ -3,17 +3,35 @@ Version History and Release Notes Releases of [SaaS Pegasus: The Django SaaS Boilerplate](https://www.saaspegasus.com/) are documented here. +## Version 2024.12 + ### UV support! -This release adds full support for [uv](https://docs.astral.sh/uv/) as a replacement package manager. -You can use uv by selecting the new "use uv" option in the project configuration page. +This release adds full support for [uv](https://docs.astral.sh/uv/) as a replacement package manager for your project. +You can use uv by selecting the new "uv" as your "Python package manager" on your project settings page. When you select uv the following changes will be made: - All requirements files are removed. -- Main project requirements will be listed in your `pyproject.toml` file. -- Pinned project requirements will be listed in a new `uv.lock` file. +- Your project requirements will now be listed in your `pyproject.toml` file. +- Your pinned project requirements will be listed in a new `uv.lock` file. - Docker containers (in development and production) will use `uv` to set up and manage the Python environment. +- A `make uv` target will be aded to Docker builds to run `uv` commands in your container. + +The main benefits of using uv are: + +- Speed. It is just way, way faster to anything related to package management. +- Easier to setup and install Python. +- Lock files (pinned versions) are cross-platform. +- More tooling. +- Speed. (It's so fast we put it twice.) + +There will be a longer write up about uv released very soon, but in the meantime you can review the updated +[python documentation](./python/setup.md) and new [uv documentation](./python/uv.md). + +The rest of the documentation has been mostly updated to accommodate uv, +though it's possible there are some places that were missed. +If you spot any issues in the docs, get in touch! ## Version 2024.11.3 From eacf69c861edeb0795e558510deaf816453b1abc Mon Sep 17 00:00:00 2001 From: Alex Roetter Date: Wed, 27 Nov 2024 16:25:50 +0100 Subject: [PATCH 14/18] Added Google OAuth instructions --- configuration.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/configuration.md b/configuration.md index bfa42df..c7f071d 100644 --- a/configuration.md +++ b/configuration.md @@ -180,6 +180,44 @@ you can follow the existing patterns and configure things based on the allauth d If you need help setting this up feel free to get in touch! Additionally see the resources below. +#### Google OAuth Specific instructions +1. Register the application with google by folllowing just the "App registration" section +[here](https://docs.allauth.org/en/latest/socialaccount/providers/google.html). Note that +the trailing slash for the "Authorized redirect URLs" is required. For example, assuming you are developing locally, +it should be set to exactly `http://localhost:8888/accounts/google/login/callback/`. +2. Add the resulting client id and secret key to the `.env` file in the root of your project. +``` + GOOGLE_CLIENT_ID="actual client id from the google console" + GOOGLE_SECRET_ID="actual secret id from the google console" +``` +3. In `/settings.py`, edit `SOCIALACCOUNT_PROVIDERS` to add `APPS` to the `"google"` section: +``` +SOCIALACCOUNT_PROVIDERS = { + "google": { + "APPS": [ + { + "client_id" : env("GOOGLE_CLIENT_ID", default=""), + "secret" : env("GOOGLE_SECRET_ID", default=""), + "key": "", + "settings": { + # You can fine tune these settings per app: + "scope": [ + "profile", + "email", + ], + "auth_params": { + "access_type": "online", + }, + }, + }, + ], + + "SCOPE": [ + # existing code... +``` + +*Note: in the future, this code might be auto generated as part of selecting "Login with Google" from the [create new project flow](https://www.saaspegasus.com/projects/create/).* + #### Social Setup Guides The Pegasus community has recommended the following guides to set things up with specific providers: From b0537db891c1c554f0b080f523cc6dc1b3f1c54f Mon Sep 17 00:00:00 2001 From: Cory Zue <66555+czue@users.noreply.github.com> Date: Thu, 28 Nov 2024 11:11:40 +0200 Subject: [PATCH 15/18] fix port --- configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configuration.md b/configuration.md index c7f071d..fcae6ba 100644 --- a/configuration.md +++ b/configuration.md @@ -184,7 +184,7 @@ Additionally see the resources below. 1. Register the application with google by folllowing just the "App registration" section [here](https://docs.allauth.org/en/latest/socialaccount/providers/google.html). Note that the trailing slash for the "Authorized redirect URLs" is required. For example, assuming you are developing locally, -it should be set to exactly `http://localhost:8888/accounts/google/login/callback/`. +it should be set to exactly `http://localhost:8000/accounts/google/login/callback/`. 2. Add the resulting client id and secret key to the `.env` file in the root of your project. ``` GOOGLE_CLIENT_ID="actual client id from the google console" From 5bd49255054fb8644a7bc8905fdb2bfe62d98841 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 28 Nov 2024 11:19:00 +0200 Subject: [PATCH 16/18] add social login + upgrade guide --- release-notes.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/release-notes.md b/release-notes.md index befa13f..bc84b80 100644 --- a/release-notes.md +++ b/release-notes.md @@ -33,6 +33,23 @@ The rest of the documentation has been mostly updated to accommodate uv, though it's possible there are some places that were missed. If you spot any issues in the docs, get in touch! +### Other updates + +- Changed the default set up of social logins to use settings-based configuration instead of `SocialApps` in the database. + **See upgrade notes if you are using social login.** + Thanks Alex for the suggestion and for helping with the updated documentation! + + +### Upgrading + +To migrate a project to `uv` see [this guide](./cookbooks.md#migrating-from-pip-tools-to-uv). + +If your application was already using social logins defined in the database, the new settings-based declaration will +conflict and cause errors on social login. +To fix this you can either delete the `APPS` section of the relevant service in `settings.SOCIALACCOUNT_PROVIDERS`, +or you can move the credentials into your project enviornment (e.g. `.env`) and delete relevant the `SocialApp` +from the Django admin. + ## Version 2024.11.3 From 08eaabb23e2d9a5033d908041d600c0ad4e71223 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Thu, 28 Nov 2024 11:25:56 +0200 Subject: [PATCH 17/18] tweak auth docs --- configuration.md | 42 +++++++++++------------------------------- 1 file changed, 11 insertions(+), 31 deletions(-) diff --git a/configuration.md b/configuration.md index fcae6ba..bab5a89 100644 --- a/configuration.md +++ b/configuration.md @@ -173,52 +173,32 @@ You'll separately need to follow the steps listed on the [provider-specific pages here](https://docs.allauth.org/en/latest/socialaccount/providers/index.html) to configure things on the other side. These steps can sometimes be a bit involved and vary by platform. +But will generally entail two steps: + +1. Creating a new application / client on the service you want to use. +2. Adding the credentials to your environment (`.env`) file. + +See the Google guide below for an example you can follow. If you want to add a social login that's not supported out of the box (e.g. Facebook/Meta or Apple), you can follow the existing patterns and configure things based on the allauth docs. If you need help setting this up feel free to get in touch! -Additionally see the resources below. +Additionally, see the resources below. #### Google OAuth Specific instructions -1. Register the application with google by folllowing just the "App registration" section + +1. Register the application with google by following just the "App registration" section [here](https://docs.allauth.org/en/latest/socialaccount/providers/google.html). Note that the trailing slash for the "Authorized redirect URLs" is required. For example, assuming you are developing locally, it should be set to exactly `http://localhost:8000/accounts/google/login/callback/`. -2. Add the resulting client id and secret key to the `.env` file in the root of your project. +2. Set the resulting client id and secret key in the `.env` file in the root of your project. ``` GOOGLE_CLIENT_ID="actual client id from the google console" GOOGLE_SECRET_ID="actual secret id from the google console" ``` -3. In `/settings.py`, edit `SOCIALACCOUNT_PROVIDERS` to add `APPS` to the `"google"` section: -``` -SOCIALACCOUNT_PROVIDERS = { - "google": { - "APPS": [ - { - "client_id" : env("GOOGLE_CLIENT_ID", default=""), - "secret" : env("GOOGLE_SECRET_ID", default=""), - "key": "", - "settings": { - # You can fine tune these settings per app: - "scope": [ - "profile", - "email", - ], - "auth_params": { - "access_type": "online", - }, - }, - }, - ], - - "SCOPE": [ - # existing code... -``` - -*Note: in the future, this code might be auto generated as part of selecting "Login with Google" from the [create new project flow](https://www.saaspegasus.com/projects/create/).* -#### Social Setup Guides +#### Other Social Setup Guides The Pegasus community has recommended the following guides to set things up with specific providers: From 726c1b83e31d09ba54df578eed6dcc17be8ba715 Mon Sep 17 00:00:00 2001 From: Cory Zue Date: Fri, 29 Nov 2024 08:37:58 +0200 Subject: [PATCH 18/18] release notes updates --- release-notes.md | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/release-notes.md b/release-notes.md index 4905f19..33019ec 100644 --- a/release-notes.md +++ b/release-notes.md @@ -5,6 +5,9 @@ Releases of [SaaS Pegasus: The Django SaaS Boilerplate](https://www.saaspegasus. ## Version 2024.12 +This release adds first-class support for using uv as a complete replacement for development and production workflows +(see below), and has a handful of fixes/changes. + ### UV support! This release adds full support for [uv](https://docs.astral.sh/uv/) as a replacement package manager for your project. @@ -12,42 +15,53 @@ You can use uv by selecting the new "uv" as your "Python package manager" on you When you select uv the following changes will be made: -- All requirements files are removed. +- All requirements.in / requirements.txt files are removed. - Your project requirements will now be listed in your `pyproject.toml` file. + - Development and production dependencies will be listed under separate dependency-groups. - Your pinned project requirements will be listed in a new `uv.lock` file. - Docker containers (in development and production) will use `uv` to set up and manage the Python environment. -- A `make uv` target will be aded to Docker builds to run `uv` commands in your container. +- A `make uv` target will be added to Docker builds to run `uv` commands in your container. The main benefits of using uv are: - Speed. It is just way, way faster to anything related to package management. - Easier to setup and install Python. -- Lock files (pinned versions) are cross-platform. +- Lock files (pinned versions) are consistent across any platform. - More tooling. - Speed. (It's so fast we put it twice.) There will be a longer write up about uv released very soon, but in the meantime you can review the updated [python documentation](./python/setup.md) and new [uv documentation](./python/uv.md). -The rest of the documentation has been mostly updated to accommodate uv, +The rest of the docs have been updated to accommodate uv, though it's possible there are some places that were missed. If you spot any issues in the docs, get in touch! +### Other fixes + +- **Upgraded the pegasus cli to fix an issue where the generated list views were not properly scoped to the + appropriate team / user.** If you used the CLI to generate any apps it's highly recommended that you check + that you are not exposing objects that should not be viewable. + ### Other updates -- Changed the default set up of social logins to use settings-based configuration instead of `SocialApps` in the database. - **See upgrade notes if you are using social login.** +- **Changed the default set up of social logins to use settings-based configuration instead of `SocialApps` in the database.** + See the upgrade notes if you are using social logins to prevent issues. Thanks Alex for the suggestion and for helping with the updated documentation! +- Updated the default flowbite setup to disable the forms plugin. This was causing styling conflicts + with the default DaisyUI styles on certain form elements, including checkboxes. +- Re-formatted the default form input template for readability. +*November 29, 2024* ### Upgrading -To migrate a project to `uv` see [this guide](./cookbooks.md#migrating-from-pip-tools-to-uv). +To migrate an existing project to `uv` see [this guide](./cookbooks.md#migrating-from-pip-tools-to-uv). If your application was already using social logins defined in the database, the new settings-based declaration will conflict and cause errors on social login. To fix this you can either delete the `APPS` section of the relevant service in `settings.SOCIALACCOUNT_PROVIDERS`, -or you can move the credentials into your project enviornment (e.g. `.env`) and delete relevant the `SocialApp` +or you can move the credentials into your project environment (e.g. `.env`) and delete relevant the `SocialApp` from the Django admin. ## Version 2024.11.3