Merge pull request #41 from saritasa-nest/feature/extend-fastapi-invo…

…cations

Add improvements after working with FastApi projects

CHANGELOG.md

@@ -6,6 +6,10 @@ We follow [Semantic Versions](https://semver.org/).
 
 - Add `django.startapp` invocation.
 - Confirm support for python 3.12.
+- Add `secrets` invocations
+- Extend `alembic` invocations to be able to make db dumps
+- Improve logic of `wait_for_database` for `alembic`
+- Add invocation for `celery` to run task
 
 ## 0.8.3
 

README.md

@@ -25,6 +25,7 @@ Collection of [invoke](https://www.pyinvoke.org/) commands used by Saritasa
   * [git](#git)
     * [git.set-git-setting](#gitset-git-setting)
     * [git.setup](#gitsetup)
+    * [git.clone-repo](#gitclone-repo)
   * [pre-commit](#pre-commit)
     * [pre-commit.install](#pre-commitinstall)
     * [pre-commit.run-hooks](#pre-commitrun-hooks)
@@ -63,8 +64,13 @@ Collection of [invoke](https://www.pyinvoke.org/) commands used by Saritasa
     * [alembic.downgrade](#alembicdowngrade)
     * [alembic.check-for-migrations](#alembiccheck-for-migrations)
     * [alembic.check-for-adjust-messages](#alembiccheck-for-adjust-messages)
+    * [alembic.load-db-dump](#alembicload-db-dump)
+    * [alembic.backup-local-db](#alembicbackup-local-db)
+    * [alembic.backup-remote-db](#alembicbackup-remote-db)
+    * [alembic.load-remote-db](#alembicload-remote-db)
   * [celery](#celery)
     * [celery.run](#celeryrun)
+    * [celery.send-task](#celerysend-task)
   * [open-api](#open-api)
     * [open-api.validate-swagger](#open-apivalidate-swagger)
   * [db](#db)
@@ -96,6 +102,8 @@ Collection of [invoke](https://www.pyinvoke.org/) commands used by Saritasa
     * [mypy](#mypyrun)
   * [pytest](#pytest)
     * [pytest](#pytestrun)
+  * [secrets](#secrets)
+    * [secrets.setup-env-credentials](#secretssetup-env-credentials)
 
 ## Installation
 
@@ -245,6 +253,10 @@ Settings:
 * `merge_ff` setting value for `merge.ff` (Default: `false`)
 * `pull_ff` setting value for `pull.ff` (Default: `only`)
 
+#### git.clone-repo
+
+Clone repo or pull latest changes to specified repo
+
 ### pre-commit
 
 #### pre-commit.install
@@ -374,7 +386,7 @@ Requires [django-extensions](https://django-extensions.readthedocs.io/en/latest/
 
 Settings:
 
-* `django_settings_path` default django settings (Default: `config.settings.local`)
+* `settings_path` default django settings (Default: `config.settings.local`)
 
 #### django.createsuperuser
 
@@ -448,7 +460,7 @@ Uses [backup_local_db](#dbbackup-local-db)
 
 Settings:
 
-* `django_settings_path` default django settings (Default: `config.settings.local`)
+* `settings_path` default django settings (Default: `config.settings.local`)
 
 #### django.backup-remote-db
 
@@ -458,7 +470,19 @@ Uses [create_dump](#db-k8screate-dump) and [get-dump](#db-k8sget-dump)
 
 Settings:
 
-* `django_settings_path` default django settings (Default: `config.settings.local`)
+* `settings_path` default django settings (Default: `config.settings.local`)
+* `remote_db_config_mapping` Mapping of db config
+  Default:
+
+  ```python
+  {
+      "dbname": "RDS_DB_NAME",
+      "host": "RDS_DB_HOST",
+      "port": "RDS_DB_PORT",
+      "username": "RDS_DB_USER",
+      "password": "RDS_DB_PASSWORD",
+  }
+  ```
 
 #### django.load-remote-db
 
@@ -469,7 +493,7 @@ Uses [create_dump](#db-k8screate-dump) and [get-dump](#db-k8sget-dump) and
 
 Settings:
 
-* `django_settings_path` default django settings (Default: `config.settings.local`)
+* `settings_path` default django settings (Default: `config.settings.local`)
 
 #### django.startapp
 
@@ -536,6 +560,111 @@ Settings:
 * `migrations_folder` migrations files location (Default: `db/migrations/versions`)
 * `adjust_messages` list of alembic adjust messages (Default: `# ### commands auto generated by Alembic - please adjust! ###`, `# ### end Alembic commands ###`)
 
+#### alembic.load-db-dump
+
+Reset db and load db dump.
+
+Uses [downgrade](#alembicdowngrade) and [load-db-dump](#dbload-db-dump)
+
+Requires [python-decouple](https://github.com/HBNetwork/python-decouple)
+
+Installed with `[env_settings]`
+
+Settings:
+
+* `db_config_mapping` Mapping of db config
+
+  Default:
+
+  ```python
+  {
+    "dbname": "rds_db_name",
+    "host": "rds_db_host",
+    "port": "rds_db_port",
+    "username": "rds_db_user",
+    "password": "rds_db_password",
+  }
+  ```
+
+#### alembic.backup-local-db
+
+Back up local db.
+
+Uses [backup_local_db](#dbbackup-local-db)
+
+Requires [python-decouple](https://github.com/HBNetwork/python-decouple)
+
+Installed with `[env_settings]`
+
+Settings:
+
+* `db_config_mapping` Mapping of db config
+
+  Default:
+
+  ```python
+  {
+    "dbname": "rds_db_name",
+    "host": "rds_db_host",
+    "port": "rds_db_port",
+    "username": "rds_db_user",
+    "password": "rds_db_password",
+  }
+  ```
+
+#### alembic.backup-remote-db
+
+Make dump of remote db and download it.
+
+Uses [create_dump](#db-k8screate-dump) and [get-dump](#db-k8sget-dump)
+
+Requires [python-decouple](https://github.com/HBNetwork/python-decouple)
+
+Installed with `[env_settings]`
+
+Settings:
+
+* `db_config_mapping` Mapping of db config
+
+  Default:
+
+  ```python
+  {
+    "dbname": "rds_db_name",
+    "host": "rds_db_host",
+    "port": "rds_db_port",
+    "username": "rds_db_user",
+    "password": "rds_db_password",
+  }
+  ```
+
+#### alembic.load-remote-db
+
+Make dump of remote db and download it and apply to local db.
+
+Uses [create-dump](#db-k8screate-dump) and [get-dump](#db-k8sget-dump) and
+[load-db-dump](#alembicload-db-dump)
+
+Requires [python-decouple](https://github.com/HBNetwork/python-decouple)
+
+Installed with `[env_settings]`
+
+Settings:
+
+* `db_config_mapping` Mapping of db config
+
+  Default:
+
+  ```python
+  {
+    "dbname": "rds_db_name",
+    "host": "rds_db_host",
+    "port": "rds_db_port",
+    "username": "rds_db_user",
+    "password": "rds_db_password",
+  }
+  ```
+
 ### celery
 
 #### celery.run
@@ -544,9 +673,21 @@ Start celery worker.
 
 Settings:
 
-* `local_cmd` command for celery (Default: `celery --app config.celery:app worker --beat --scheduler=django --loglevel=info`)
+* `app` path to app  (Default: `config.celery.app`)
+* `scheduler` scheduler (Default: `django`)
+* `loglevel` log level for celery (Default: `info`)
+* `extra_params` extra params for worker (Default: `("--beat",)`)
+* `local_cmd` command for celery (Default: `celery --app {app} worker --scheduler={scheduler} --loglevel={info} {extra_params}`)
 * `service_name` name of celery service (Default: `celery`)
 
+#### celery.send-task
+
+Send task to celery worker.
+
+Settings:
+
+* `app` path to app  (Default: `config.celery.app`)
+
 ### open-api
 
 #### open-api.validate-swagger
@@ -749,3 +890,19 @@ Run pytest in `path` with `params`.
 Settings:
 
 * `pytest_entry` python entry command (Default: `-m pytest`)
+
+### secrets
+
+#### secrets.setup-env-credentials
+
+Fill specified credentials in your file from k8s.
+This invocations downloads `.env` file from pod in k8s.
+It will replace specified credentials(`--credentials`) in
+specified file `.env` file (`--env_file_path` or `.env` as default)
+
+Requires [python-decouple](https://github.com/HBNetwork/python-decouple)
+
+Settings for k8s:
+
+* `secret_file_path_in_pod` path to secret in pod (**REQUIRED**)
+* `temp_secret_file_path` path for temporary file (Default: `.env.to_delete`)

saritasa_invocations/__init__.py

@@ -17,6 +17,7 @@
     pre_commit,
     pytest,
     python,
+    secrets,
     system,
 )
 from saritasa_invocations._config import (

saritasa_invocations/_config.py

@@ -120,10 +120,14 @@ class DjangoSettings:
 class CelerySettings:
     """Settings for celery module."""
 
+    app: str = "config.celery.app"
+    scheduler: str = "django"
     service_name: str = "celery"
+    loglevel: str = "info"
+    extra_params: tuple[str] = ("--beat",)
     local_cmd: str = (
-        "celery --app config.celery:app "
-        "worker --beat --scheduler=django --loglevel=info"
+        "celery --app {app} "
+        "worker --scheduler={scheduler} --loglevel={info} {extra_params}"
     )
 
 
@@ -150,6 +154,15 @@ class AlembicSettings:
         "# ### commands auto generated by Alembic - please adjust! ###",
         "# ### end Alembic commands ###",
     )
+    db_config_mapping: dict[str, str] = dataclasses.field(
+        default_factory=lambda: {
+            "dbname": "rds_db_name",
+            "host": "rds_db_host",
+            "port": "rds_db_port",
+            "username": "rds_db_user",
+            "password": "rds_db_password",
+        },
+    )
 
 
 @dataclasses.dataclass
@@ -249,6 +262,8 @@ class K8SSettings(metaclass=K8SSettingsMeta):
     default_entry: str | None = None
     python_shell: str | None = None
     health_check: str | None = None
+    secret_file_path_in_pod: str | None = None
+    temp_secret_file_path: str | None = None
     env_color: str | None = None
 
 
@@ -270,6 +285,8 @@ class K8SDefaultSettings:
     default_entry: str = "cnb/lifecycle/launcher bash"
     python_shell: str = "shell_plus"
     health_check: str = "health_check"
+    secret_file_path_in_pod: str | None = None
+    temp_secret_file_path: str = ".env.to_delete"
     env_color: str = "cyan"
 
 
@@ -290,6 +307,8 @@ class K8SGeneratedSettings:
     default_entry: str
     python_shell: str
     health_check: str
+    secret_file_path_in_pod: str
+    temp_secret_file_path: str
     env_color: str
 
     @classmethod