Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GH-171] - Update deploy.yml to Send Notifications to Microsoft Teams and Align with Other Repos #173

Merged
merged 21 commits into from
Jan 21, 2025
Merged
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
df5a7d7
fix(ci): Fix create-release.yml workflow
soufianerafik Jan 17, 2025
488d8ba
fix(ci): Removing GH-169 test branch from list of branches
soufianerafik Jan 17, 2025
9045fd4
fix(ci): Moving deploy.yml notifications to teams and aligning it wit…
soufianerafik Jan 17, 2025
329546e
fix(ci): Fix identation on deploy.yml
soufianerafik Jan 17, 2025
a2da962
fix(ci): Fix associated record error
soufianerafik Jan 17, 2025
2fabf4d
Merge branch 'main' into GH-171
soufianerafik Jan 17, 2025
2cb0244
fix(ci): Fix associated script not found error
soufianerafik Jan 17, 2025
99b6eae
fix(ci): Add new teams webhook UW_DIRECTORY_DEPLOY_MS_TEAMS_WEBHOOK_URL
soufianerafik Jan 17, 2025
2aea710
fix(ci): Debug Failing Action
soufianerafik Jan 17, 2025
8eac84e
fix(ci): Install poetry with pip
soufianerafik Jan 17, 2025
0a6341e
fix(ci): Add "Run Poetry Install" script
soufianerafik Jan 17, 2025
ff4eb32
fix(ci): Remove unused line
soufianerafik Jan 17, 2025
b9b63de
fix(ci): Fix typo
soufianerafik Jan 17, 2025
bf80495
fix(ci): Add "Set up Python 3.10" step
soufianerafik Jan 17, 2025
f340129
fix(ci): Add '--no-root'
soufianerafik Jan 17, 2025
0f866cf
Merge remote-tracking branch 'origin/GH-171' into GH-171
soufianerafik Jan 17, 2025
d19588c
fix(ci): Revert '--no-root' change
soufianerafik Jan 21, 2025
850da26
fix(ci): Fix failing action
soufianerafik Jan 21, 2025
6993300
fix(ci): Remove --no-root
soufianerafik Jan 21, 2025
b96f621
fix(ci): Add --no-root
soufianerafik Jan 21, 2025
ebe07cb
fix(ci): Skip Notifications to dev instances
soufianerafik Jan 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
303 changes: 182 additions & 121 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,38 +3,43 @@ name: Deploy the UW Directory
on:
workflow_dispatch:
inputs:
target_cluster:
description: cluster. Choose from dev/eval/prod.
default: eval
target-stage:
required: true
rfc:
default: eval
description: >
rfc. The RFC number or link associated with this
deployment. Required when deploying to prod.
(target-stage)
Which cluster you want to deploy to.
Choose from dev, eval, prod.
target-version:
required: false
description: >
(target-version)
The semver you want to deploy. If you do not provide this, the workflow
will promote from the "previous" cluster (dev -> eval, eval -> prod).
associated-record:
required: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference #identity-uw

description: >
version. The version to deploy (e.g., '1.2.3'). If not provided,
the most recent release candidate will be used (eval will source from dev,
prod will source from eval).
(associated-record)
Only required if target-stage is prod.
A link to an RFC, Jira, or other document associated
with this change.

env:
GCLOUD_TOKEN: ${{ secrets.GCR_TOKEN }}
SLACK_BOT_TOKEN: ${{ secrets.ACTIONS_SLACK_BOT_TOKEN }}
DRY_RUN: false # ?
STEP_SCRIPTS: ${{ github.workspace }}/.github/steps/deploy
# target_cluster: dev
UW_DIRECTORY_DEPLOY_MS_TEAMS_WEBHOOK_URL: ${{ secrets.UW_DIRECTORY_DEPLOY_MS_TEAMS_WEBHOOK_URL }}

jobs:
# The `configure` job reconciles the target version (if it wasn't explicitly provided),
# creates a slack notification for the deployment (except for developer instances),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"creates a slack notification for the deployment (except for developer instances)"
did we keep the part about "except for developer instances"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, great call @goulter - Thanks!

It's handled here: https://github.com/UWIT-IAM/uw-husky-directory/pull/173/files#diff-28802fbf11c83a2eee09623fb192785e7ca92a3f40602a517c011b947a1822d3L167

+      - name: Notify Teams of Deployment Start
+        # creates MS Teams notification for the deployment (except for developer instances),
+       if: env.target_stage == 'dev' || env.target_stage == 'eval' || env.target_stage == 'prod'

Just to confirm, do we have developer instances like identity.UW? Either way, this won't send notifications if we create them in the future. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, thats a great point. There are no developer instances of the directory.

# and updates the slack notification with additional deployment context information.
# The `configure` job reconciles the target version (if it wasn't explicitly provided)
# and prepares the deployment context. Additionally, a Teams notification is sent at the
# start of the deployment in the `deploy` job, and another notification is sent upon
# deployment completion (regardless of success or failure) in the `notify-teams-finish` job.
configure:
env:
target_cluster: ${{ github.event.inputs.target_cluster }}
rfc: ${{ github.event.inputs.rfc }}
target_stage: ${{ github.event.inputs.target-stage }}
target_version: ${{ github.event.inputs.target-version }}
associated_record: ${{ github.event.inputs.associated-record }}
outputs:
target-version: ${{ steps.reconcile-version.outputs.target-version }}
slack-notification-id: ${{ steps.slack.outputs.canvas-id }} #?
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand All @@ -43,26 +48,28 @@ jobs:
# currently validate this record, because endpoints usually require
# authentication.
- name: Verify prod deployment record
if: github.event.inputs.target_cluster == 'prod'
if: github.event.inputs.target-stage == 'prod'
run: |
if [[ -z "${{ env.rfc }}" ]]
if [[ -z "${{ env.associated_record }}" ]]
then
echo "Deployment to prod requires a link to an rfc"
echo "Deployment to prod requires a link to an associated record"
exit 1
fi
# If the entity who created this deployment did not provide a version,
# we will derive the version from the target stage's "previous" environment.
# This means that deployments to eval will promote from dev, and deployments
# to prod will promote from eval. It's OK if the same version is re-deployed,
# it will have the same net effect of a `kubectl rollout restart`.
# If the entity does provide a semver (target_version), that is used.

# Output: `target-version`: The reconciled version to deploy.
- name: Reconcile target deployment version
id: reconcile-version
run: |
set -x
if [[ -z "${target_version}" ]]
if [[ -z "${{ env.target_version }}" ]]
then
case "${target_cluster}" in
case "${target_stage}" in
prod)
source_stage=eval
;;
Expand All @@ -73,73 +80,20 @@ jobs:
source_stage=dev
;;
esac
target_version=$(./scripts/get-deployed-version.sh -s ${target_cluster})
fi
if [[ "${source_stage}" == "poetry" ]]
then
# Get the target version from the poetry configuration
source ./scripts/globals.sh
target_version=$(get_poetry_version)
target_version=$(./scripts/get-deployed-version.sh -s ${target_stage})

if [[ "${source_stage}" == "poetry" ]]
then
source ./scripts/globals.sh
target_version=$(get_poetry_version)
else
target_version=$(./scripts/get-deployed-version.sh -s ${source_stage})
fi
else
target_version=$(./scripts/get-deployed-version.sh -s ${source_stage})
target_version="${{ env.target_version }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you clarify the change? (lines 84/93)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@goulter : It is used to directly set the target_version when it is explicitly provided as input (via the target-version environment variable).

For more details please check https://github.com/UWIT-IAM/identity-uw/blob/develop/.github/workflows/deploy-from-ui.yml#L98

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have been more specific. Why not keep target_version=$(./scripts/get-deployed-version.sh -s ${source_stage})?

Copy link
Contributor Author

@soufianerafik soufianerafik Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change from target_version=$(./scripts/get-deployed-version.sh -s ${source_stage}) to target_version="${{ env.target_version }}" in the else clause is intentional.

In this case, we've already confirmed that env.target_version exists (since we're in the else block of if [[ -z "${{ env.target_version }}" ]]), so we should use the provided version directly rather than pulling from the source stage. This is done to make sure that when a user explicitly provides a version, we respect that choice instead of overwriting it with a version from another environment.

The original code would ignore the user-provided version and always pull from the source stage, which defeats the purpose of allowing version specification through the input variable.

This change allows for two clear paths:

  1. No version provided -> derive from source stage.
  2. Version provided -> use that version directly.

I've also made this diagram to help visually understand the logic:

Screenshot 2025-01-21 at 2 04 30 PM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so, the existing code was broke, lovely. Thanks for fixing it.

fi
echo "target-version=${target_version}" >> $GITHUB_OUTPUT
# For shared instances (dev, eval, prod) we create a slack notification
# so that others can be aware of the change.
# TODO: Eval and prod deployment notifications should go to #iam-uwnetid
# https://github.com/UWIT-IAM/uw-husky-directory/issues/152
- name: Create notification
id: slack
# Don't send notifications for developer instances
if: env.target_cluster == 'dev' || env.target_cluster == 'eval' || env.target_cluster == 'prod'
uses: UWIT-IAM/actions/[email protected]
env:
target_version: ${{ steps.reconcile-version.outputs.target-version }}
with:
json: >
{
"description": "Deploy UW Husky Directory v${{ env.target_version }} to ${{ env.target_cluster }}",
"channel": "#iam-bots",
"status": "in progress",
"steps": [
{
"stepId": "deploy",
"status": "in progress",
"description": "Create deployment and wait for update"
}
]
}
# This adds a special link for associated records, when they are provided;
# this logic was a little too complex to capture only using github actions contexts,
# so needed to be its own li'l step.
# Output: context - The slack message snippet that provides a link to the audit record.
- if: env.rfc
id: audit
run: |
slack_link="<${{ env.rfc }} | Audit Record>"
echo "context=[${slack_link}]" >> $GITHUB_OUTPUT

# If we have a Slack notification for this change, add a context artifact to it.
# This provides at-a-glance links and details in the slack notification for
# traceability.
# The output winds up reading something like:
# "Deployment workflow for UW Husky Directory app image started by goulter [Audit Record]" where
# all of "deployment workflow" "app image" "goulter" and "Audit Record" are all hyperlinks.
- if: steps.slack.outputs.canvas-id
uses: UWIT-IAM/actions/[email protected]
env:
workflow_link: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
image_link: https://gcr.io/uwit-mci-iam/husky-directory.app:${{ steps.reconcile-version.outputs.target-version }}
target_version: ${{ steps.version.outputs.target-version }}
actor_link: https://github.com/${{ github.actor }}
with:
canvas-id: ${{ steps.slack.outputs.canvas-id }}
command: add-artifact
description: >
<${{ env.workflow_link }} | Deployment workflow> for UW Husky Directory
<${{ env.image_link }} | app image ${{ env.target_version }}>
started by <${{ env.actor_link }} | ${{ github.actor }}>
${{ steps.audit.outputs.context }}
echo "***target_version=${target_version}" >> $GITHUB_OUTPUT
echo "::set-output name=target-version::${target_version}"

# The deploy job simply runs the deploy script. This script will wait for
# deployments to complete before exiting. If the deployment times out,
Expand All @@ -155,52 +109,159 @@ jobs:
cancel-in-progress: false
env:
target_version: ${{ needs.configure.outputs.target-version }}
target_stage: ${{ github.event.inputs.target_cluster }}
associated_record: ${{ github.event.inputs.rfc }}
slack_notification_id: ${{ needs.configure.outputs.slack-notification-id }}
target_stage: ${{ github.event.inputs.target-stage }}
associated_record: ${{ github.event.inputs.associated-record }}
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v3
- uses: abatilo/[email protected]
- run: |
- name: Notify Teams of Deployment Start
run: |
associated_record="${{ github.event.inputs.associated-record }}"
target_stage="${{ github.event.inputs.target-stage }}"

# Determine associated record value based on target stage and input
if [ -z "$associated_record" ]; then
if [ "$target_stage" != "prod" ]; then
associated_record="Not required (deployment to $target_stage)"
else
associated_record="Missing (required for production)"
fi
fi

curl -H "Content-Type: application/json" \
-d '{
"type": "message",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.adaptive",
"content": {
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.4",
"body": [
{
"type": "TextBlock",
"size": "Large",
"weight": "Bolder",
"text": "UW Directory - Deployment Notification"
},
{
"type": "TextBlock",
"text": "Deployment to stage **${{ github.event.inputs.target-stage }}** is starting.",
"wrap": true
},
{
"type": "FactSet",
"facts": [
{"title": "Stage:", "value": "${{ github.event.inputs.target-stage }}"},
{"title": "Version:", "value": "${{ needs.configure.outputs.target-version }}"},
{"title": "Associated Record:", "value": "'"${associated_record}"'"},
{"title": "Initiated By:", "value": "${{ github.actor }}"}
]
}
]
}
}
]
}' \
"${{ env.UW_DIRECTORY_DEPLOY_MS_TEAMS_WEBHOOK_URL }}"

- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install Poetry with pip
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry --version

- name: Run Poetry Install
run: |
sudo apt-get -y install jq
poetry install
poetry install --no-root

- uses: actions/checkout@v3
- uses: ./.github/actions/configure-docker
with:
project-name: ${{ secrets.IAM_GCR_REPO }}
gcr-token: ${{ secrets.GCR_TOKEN }}
- run: |
gcloud auth activate-service-account --key-file=${GOOGLE_APPLICATION_CREDENTIALS}
./scripts/deploy.sh -g -t ${target_stage} -v ${target_version} -r "${{ github.event.inputs.rfc }}"
./scripts/deploy.sh -g -t ${target_stage} -v ${target_version} -r "${{ github.event.inputs.associated-record }}"
# probably need to account for -x, --dry-run at some point. lets see how things go.

# This removes extraneous volatile information from the Slack notification,
# leaving only any errors, the final status, and the deployment context artifact.
cleanup:
notify-teams-finish:
runs-on: ubuntu-latest
needs: [configure, deploy]
permissions:
contents: read
id-token: write
if: needs.configure.outputs.slack-notification-id
env:
SLACK_CANVAS_ID: ${{ needs.configure.outputs.slack-notification-id }}
deploy_result: ${{ needs.deploy.result == 'success' && 'succeeded' || 'failed' }}
needs: [ configure, deploy ]
steps:
- uses: actions/checkout@v3
- uses: ./.github/actions/configure-docker
with:
project-name: ${{ secrets.IAM_GCR_REPO }}
gcr-token: ${{ secrets.GCR_TOKEN }}
- uses: UWIT-IAM/actions/[email protected]
with:
command: update-workflow
step-id: deploy
step-status: ${{ env.deploy_result }}
canvas-id: ${{ needs.configure.outputs.slack-notification-id }}
- uses: UWIT-IAM/actions/[email protected]
with:
workflow-status: ${{ env.deploy_result }}
- name: Notify Teams of Deployment Completion
if: always() # This step runs regardless of success or failure
run: |
deploy_result="${{ needs.deploy.result }}"
deploy_status="Failed"
associated_record="${{ github.event.inputs.associated-record }}"
target_stage="${{ github.event.inputs.target-stage }}"

if [ "$deploy_result" == "success" ]; then
deploy_status="Succeeded"
fi

# Determine associated record value based on target stage and input
if [ -z "$associated_record" ]; then
if [ "$target_stage" != "prod" ]; then
associated_record="Not required (deployment to $target_stage)"
else
associated_record="Missing (required for production)"
fi
fi

curl -H "Content-Type: application/json" \
-d '{
"type": "message",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.adaptive",
"content": {
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.4",
"body": [
{
"type": "TextBlock",
"size": "Large",
"weight": "Bolder",
"text": "UW Directory - Deployment Notification"
},
{
"type": "TextBlock",
"text": "Deployment to stage **${{ github.event.inputs.target-stage }}** has **'"$deploy_status"'**.",
"wrap": true
},
{
"type": "FactSet",
"facts": [
{"title": "Stage:", "value": "${{ github.event.inputs.target-stage }}"},
{"title": "Version:", "value": "${{ needs.configure.outputs.target-version }}"},
{"title": "Associated Record:", "value": "'"${associated_record}"'"},
{"title": "Initiated By:", "value": "${{ github.actor }}"},
{"title": "Status:", "value": "'"$deploy_status"'"}
]
}
],
"actions": [
{
"type": "Action.OpenUrl",
"title": "View Workflow",
"url": "https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
}
]
}
}
]
}' \
"${{ env.UW_DIRECTORY_DEPLOY_MS_TEAMS_WEBHOOK_URL }}"

Loading