diff --git a/.github/workflows/publish-operator-hub.yaml b/.github/workflows/publish-operator-hub.yaml index b02dc1088..822d00f5a 100644 --- a/.github/workflows/publish-operator-hub.yaml +++ b/.github/workflows/publish-operator-hub.yaml @@ -64,7 +64,7 @@ jobs: fetch-tags: true ref: ${{ env.VERSION }} path: awx-operator-${{ env.VERSION }} - fetch-depth: 0 # fetch all history so that git describe works + fetch-depth: 0 # fetch all history so that git describe works - name: Copy scripts to awx-operator-${{ env.VERSION }} run: | diff --git a/.helm/starter/README.md b/.helm/starter/README.md index 563029103..1250b7c7f 100644 --- a/.helm/starter/README.md +++ b/.helm/starter/README.md @@ -32,6 +32,63 @@ These sub-headers aim to be a more intuitive entrypoint into customizing your de ### External Postgres The `AWX.postgres` section simplifies the creation of the external postgres secret. If enabled, the configs provided will automatically be placed in a `postgres-config` secret and linked to the `AWX` resource. For proper secret management, the `AWX.postgres.password` value, and any other sensitive values, can be passed in at the command line rather than specified in code. Use the `--set` argument with `helm install`. Supplying the password this way is not recommended for production use, but may be helpful for initial PoC. +### Additional Kubernetes Resources +The `AWX.extraDeploy` section allows the creation of additional Kubernetes resources. This simplifies setups requiring additional objects that are used by AWX, e.g. using `ExternalSecrets` to create Kubernetes secrets. + +Resources are passed as an array, either as YAML or strings (literal "|"). The resources are passed through `tpl`, so templating is possible. Example: + +```yaml +AWX: + # enable use of awx-deploy template + ... + + # configurations for external postgres instance + postgres: + enabled: false + ... + +extraDeploy: + - | + apiVersion: external-secrets.io/v1beta1 + kind: ExternalSecret + metadata: + name: {{ .Release.Name }}-postgres-secret-string-example + namespace: {{ .Release.Namespace }} + labels: + app: {{ .Release.Name }} + spec: + secretStoreRef: + name: vault + kind: ClusterSecretStore + refreshInterval: "1h" + target: + name: postgres-configuration-secret-string-example + creationPolicy: "Owner" + deletionPolicy: "Delete" + dataFrom: + - extract: + key: awx/postgres-configuration-secret + + - apiVersion: external-secrets.io/v1beta1 + kind: ExternalSecret + metadata: + name: "{{ .Release.Name }}-postgres-secret-yaml-example" + namespace: "{{ .Release.Namespace }}" + labels: + app: "{{ .Release.Name }}" + spec: + secretStoreRef: + name: vault + kind: ClusterSecretStore + refreshInterval: "1h" + target: + name: postgres-configuration-secret-yaml-example + creationPolicy: "Owner" + deletionPolicy: "Delete" + dataFrom: + - extract: + key: awx/postgres-configuration-secret +``` ## Values Summary @@ -43,6 +100,10 @@ The `AWX.postgres` section simplifies the creation of the external postgres secr | `AWX.spec` | specs to directly configure the AWX resource | `{}` | | `AWX.postgres` | configurations for the external postgres secret | - | +### extraDeploy +| Value | Description | Default | +|---|---|---| +| `extraDeploy` | array of additional resources to be deployed (supports YAML or literal "\|") | - | # Contributing @@ -63,5 +124,3 @@ The chart is currently hosted on the gh-pages branch of the repo. During the rel Instead of CR, we use `helm repo index` to generate an index from all locally pulled chart versions. Since we build from scratch every time, the timestamps of all entries will be updated. This could be improved by using yq or something similar to detect which tags are already in the index.yaml file, and only merge in tags that are not present. Not using CR could be addressed in the future by keeping the chart built as a part of releases, as long as CR compares the chart to previous release packages rather than previous commits. If the latter is the case, then we would not have the necessary history for comparison. - - diff --git a/.helm/starter/templates/extra-list.yaml b/.helm/starter/templates/extra-list.yaml new file mode 100644 index 000000000..5237334f8 --- /dev/null +++ b/.helm/starter/templates/extra-list.yaml @@ -0,0 +1,8 @@ +{{- range .Values.extraDeploy }} +--- + {{- if typeIs "string" . }} + {{- tpl . $ }} + {{- else }} + {{- tpl (. | toYaml | nindent 0) $ }} + {{- end }} +{{- end }} diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 71948e7d6..9134d7cfb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -25,7 +25,7 @@ Have questions about this document or anything not covered here? Please file a n ## Submmiting your work -1. From your fork `devel` branch, create a new brach to stage your changes. +1. From your fork `devel` branch, create a new branch to stage your changes. ```sh #> git checkout -b ``` diff --git a/docs/upgrade/upgrading.md b/docs/upgrade/upgrading.md index 8bd75107f..e826caded 100644 --- a/docs/upgrade/upgrading.md +++ b/docs/upgrade/upgrading.md @@ -1,6 +1,10 @@ ### Upgrading -To upgrade AWX, it is recommended to upgrade the awx-operator to the version that maps to the desired version of AWX. To find the version of AWX that will be installed by the awx-operator by default, check the version specified in the `image_version` variable in `roles/installer/defaults/main.yml` for that particular release. +To upgrade AWX, it is recommended to upgrade the awx-operator to the version that maps to the desired version of AWX. To find the version of AWX that will be installed by the awx-operator by default, check the version specified in the `DEFAULT_AWX_VERSION` variable for that particular release. You can do so by running the following command +```shell +AWX_OPERATOR_VERSION=2.8.0 +docker run --entrypoint="" quay.io/ansible/awx-operator:$AWX_OPERATOR_VERSION bash -c "env | grep DEFAULT_AWX_VERSION" +``` Apply the awx-operator.yml for that release to upgrade the operator, and in turn also upgrade your AWX deployment. diff --git a/docs/user-guide/advanced-configuration/scaling-the-web-and-task-pods-independently.md b/docs/user-guide/advanced-configuration/scaling-the-web-and-task-pods-independently.md index f7ccf76dc..1dc9c3440 100644 --- a/docs/user-guide/advanced-configuration/scaling-the-web-and-task-pods-independently.md +++ b/docs/user-guide/advanced-configuration/scaling-the-web-and-task-pods-independently.md @@ -5,4 +5,4 @@ You can scale replicas up or down for each deployment by using the `web_replicas - If you specify the `replicas` field, the key passed will scale both the `web` and `task` replicas to the same number. - If `web_replicas` or `task_replicas` is ever passed, it will override the existing `replicas` field on the specific deployment with the new key value. -These new replicas can be constrained in a similar manner to previous single deployments by appending the particular deployment name in front of the constraint used. More about those new constraints can be found below in the [Assigning AWX pods to specific nodes](#assigning-awx-pods-to-specific-nodes) section. +These new replicas can be constrained in a similar manner to previous single deployments by appending the particular deployment name in front of the constraint used. More about those new constraints can be found in the [Assigning AWX pods to specific nodes](./assigning-awx-pods-to-specific-nodes.md) page. diff --git a/roles/backup/tasks/postgres.yml b/roles/backup/tasks/postgres.yml index bb1812a01..b885abbf3 100644 --- a/roles/backup/tasks/postgres.yml +++ b/roles/backup/tasks/postgres.yml @@ -70,13 +70,6 @@ command: >- touch {{ backup_dir }}/tower.db -- name: Set permissions on file for database dump - k8s_exec: - namespace: "{{ backup_pvc_namespace }}" - pod: "{{ ansible_operator_meta.name }}-db-management" - command: >- - bash -c "chmod 660 {{ backup_dir }}/tower.db && chown :root {{ backup_dir }}/tower.db" - - name: Set full resolvable host name for postgres pod set_fact: resolvable_db_host: '{{ (awx_postgres_type == "managed") | ternary(awx_postgres_host + "." + ansible_operator_meta.namespace + ".svc", awx_postgres_host) }}' # yamllint disable-line rule:line-length diff --git a/roles/installer/tasks/install.yml b/roles/installer/tasks/install.yml index b3795dff2..2398ebb4d 100644 --- a/roles/installer/tasks/install.yml +++ b/roles/installer/tasks/install.yml @@ -84,8 +84,8 @@ namespace: "{{ ansible_operator_meta.namespace }}" register: restore_status_check until: - # yamllint disable-line rule:line-length - - (restore_status_check.resources | length == 0) or (restore_status_check.resources | selectattr('spec.deployment_name', 'equalto', ansible_operator_meta.name) | map(attribute='status') | selectattr('restoreComplete', 'defined') | map(attribute='restoreComplete') | list | length > 0) + # yamllint disable-line rule:line-length + - (restore_status_check.resources | length == 0) or (restore_status_check.resources | selectattr('spec.deployment_name', 'equalto', ansible_operator_meta.name) | map(attribute='status') | selectattr('restoreComplete', 'defined') | map(attribute='restoreComplete') | list | length > 0) delay: 10 retries: 8640 ignore_errors: yes diff --git a/roles/installer/templates/configmaps/config.yaml.j2 b/roles/installer/templates/configmaps/config.yaml.j2 index 649f3dd4f..b4536b39f 100644 --- a/roles/installer/templates/configmaps/config.yaml.j2 +++ b/roles/installer/templates/configmaps/config.yaml.j2 @@ -54,9 +54,6 @@ data: INTERNAL_API_URL = 'http://127.0.0.1:8052' - # Sets Ansible Collection path - AWX_ANSIBLE_COLLECTIONS_PATHS = '/var/lib/awx/vendor/awx_ansible_collections' - # Container environments don't like chroots AWX_PROOT_ENABLED = False diff --git a/roles/restore/tasks/postgres.yml b/roles/restore/tasks/postgres.yml index 36a538089..617a23ace 100644 --- a/roles/restore/tasks/postgres.yml +++ b/roles/restore/tasks/postgres.yml @@ -82,25 +82,34 @@ -p {{ awx_postgres_port }} no_log: "{{ no_log }}" -- name: Set drop db command - set_fact: - pg_drop_db: >- - echo 'DROP DATABASE {{ awx_postgres_database }} WITH (FORCE);' | PGPASSWORD='{{ awx_postgres_pass }}' psql - -U {{ awx_postgres_user }} - -h {{ resolvable_db_host }} - -d postgres - -p {{ awx_postgres_port }} - no_log: "{{ no_log }}" +- name: Force drop and create database if force_drop_db is true + block: + - name: Set drop db command + set_fact: + pg_drop_db: >- + echo 'DROP DATABASE {{ awx_postgres_database }} WITH (FORCE);' | PGPASSWORD='{{ awx_postgres_pass }}' psql + -U {{ awx_postgres_user }} + -h {{ resolvable_db_host }} + -d postgres + -p {{ awx_postgres_port }} ; + no_log: "{{ no_log }}" -- name: Set create db command - set_fact: - pg_create_db: >- - echo 'CREATE DATABASE {{ awx_postgres_database }} WITH OWNER = {{ awx_postgres_user }};' | PGPASSWORD='{{ awx_postgres_pass }}' psql - -U {{ awx_postgres_user }} - -h {{ resolvable_db_host }} - -d postgres - -p {{ awx_postgres_port }} - no_log: "{{ no_log }}" + - name: Set create db command + set_fact: + pg_create_db: >- + echo 'CREATE DATABASE {{ awx_postgres_database }} WITH OWNER = {{ awx_postgres_user }};' | PGPASSWORD='{{ awx_postgres_pass }}' psql + -U {{ awx_postgres_user }} + -h {{ resolvable_db_host }} + -d postgres + -p {{ awx_postgres_port }} ; + no_log: "{{ no_log }}" + + - name: Set complete pg restore command + set_fact: + pg_drop_create: >- + {{ pg_drop_db }} + {{ pg_create_db }} + when: force_drop_db - name: Restore database dump to the new postgresql container k8s_exec: @@ -124,14 +133,11 @@ trap 'end_keepalive \"$keepalive_file\" \"$keepalive_pid\"' EXIT SIGINT SIGTERM echo keepalive_pid: $keepalive_pid set -e -o pipefail - if {{ force_drop_db }}; then - {{ pg_drop_db }} - {{ pg_create_db }} - fi + {{ pg_drop_create }} cat {{ backup_dir }}/tower.db | PGPASSWORD='{{ awx_postgres_pass }}' {{ pg_restore }} + PG_RC=$? set +e +o pipefail - echo 'Successful' + exit $PG_RC " register: data_migration no_log: "{{ no_log }}" - failed_when: "'Successful' not in data_migration.stdout" diff --git a/roles/restore/vars/main.yml b/roles/restore/vars/main.yml index 94ccf9de0..2495549f7 100644 --- a/roles/restore/vars/main.yml +++ b/roles/restore/vars/main.yml @@ -14,3 +14,7 @@ broadcast_websocket_secret: '{{ deployment_name }}-broadcast-websocket' postgres_configuration_secret: '{{ deployment_name }}-postgres-configuration' supported_pg_version: 13 image_pull_policy: IfNotPresent + +# If set to true, the restore process will delete the existing database and create a new one +force_drop_db: false +pg_drop_create: ''