Skip to content

Commit

Permalink
feat: create github action to trigger swithcover countdown
Browse files Browse the repository at this point in the history
  • Loading branch information
thegentlemanphysicist committed Jun 4, 2024
1 parent f23b2e3 commit 3d31ef4
Show file tree
Hide file tree
Showing 3 changed files with 128 additions and 1 deletion.
56 changes: 56 additions & 0 deletions .github/workflows/schedule-preemptive-failove.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Schedule Preemptive Failover

on:
workflow_dispatch:
inputs:
project:
description: "The target project"
type: choice
required: true
options: ["SANDBOX", "PRODUCTION"]
default: "SANDBOX"
environment:
description: "env to deploy"
type: choice
options: ["dev","test","prod"]
default: "dev"
timestart:
description: "The time to trigger the preemptive failover (24 hour PST)"
type: string
required: true
default: "YYYY/MM/DD HH:MM"
timeend:
description: "The time to switch traffic back to Gold (24 hour PST)"
type: string
required: true
default: "YYYY/MM/DD HH:MM"

jobs:
transition-scripts:
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- name: Set the deployment namespace
run: |
chmod +x ./namespace-setter.sh
./namespace-setter.sh ${{ github.event.inputs.project }} ${{ github.event.inputs.environment }}
working-directory: .github/helpers

- name: Verify Time Inputs

- name: Login Openshift Gold & Golddr
uses: ./.github/actions/oc-login
with:
namespace: $NAMESPACE
oc-server-gold: ${{ secrets.OPENSHIFT_SERVER_GOLD }}
oc-token-gold: ${{ secrets.OPENSHIFT_TOKEN_GOLD }}
oc-server-golddr: ${{ secrets.OPENSHIFT_SERVER_GOLDDR }}
oc-token-golddr: ${{ secrets.OPENSHIFT_TOKEN_GOLDDR }}

- name: Run transition script
run: |
echo "Running on the $NAMESPACE namespace"
chmod +x ./deploy-switchover-agent.sh
./deploy-switchover-agent.sh $NAMESPACE "${{ github.event.inputs.timestart }}" "${{ github.event.inputs.timeend }}"
working-directory: transition-scripts
2 changes: 1 addition & 1 deletion docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The CSS app should be put in maintenance mode when the production environent is

The switchover agent is able to create and close incidents on uptime status pages. (Production is located at [https://status.loginproxy.gov.bc.ca/](https://status.loginproxy.gov.bc.ca/)). To do this, two environment vars must be configured:

- UPTIME_STATUS_PAGE_ID: This integer can be found in the status page's non-vanity url for the statu page hosted by uptime.com.
- UPTIME_STATUS_PAGE_ID: This integer can be found in the status page's non-vanity url for the status page hosted by uptime.com.
- UPTIME_STATUS_TOKEN: The credential used for the uptime.com API. It can be found [here](https://uptime.com/api/tokens).
## Preemptive Failover

Expand Down
71 changes: 71 additions & 0 deletions transition-scripts/deploy-switchover-agent.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/bin/bash
set -e

usage() {
cat <<EOF
Redeploy the switchover agent in <namespace> with time an optional preemptive failover failback argument.
Usages:
$0 <namespace> <time_start> <time_end>
Available namespaces:
- e4ca1d-dev
- e4ca1d-test
- e4ca1d-prod
- eb75ad-dev
- eb75ad-test
- eb75ad-prod
Time format is "YYYY/MM/DD HH:MM". There must be 1 or 3 arguments supplied for the script to run.
Examples:
$ $0 e4ca1d-dev "1984/12/12 17:30" "2001/12/12 03:30"
EOF
}

if [[ "$#" -ne 1 && "$#" -ne 3 ]]; then
usage
exit 1
fi

namespace=$1
secret_name="sso-switchover-agent"
pwd="$(dirname "$0")"
source "$pwd/helpers/_all.sh"


validate_time_regex() {
local input="$1"
#TODO IMPROVE THIS REGEX
local regex="^2[0-9]{3}\/[0-9]{2}\/[0-9]{2}\s[0-9]{2}:[0-9]{2}$"
if [[ "$input" =~ $regex ]]; then
echo "The input matches the time regex."
return 0
else
echo "The input does not match the time regex."
exit 1
fi
}


# Switchover Agent deployment is in the Golddr cluster
switch_kube_context "golddr" "$namespace"
check_ocp_cluster "golddr"

if [[ "$#" -eq 3 ]]; then
time_start=$2
time_end=$3
validate_time_regex "$time_start"
validate_time_regex "$time_end"

kubectl -n "$namespace" patch secret "$secret_name" \
--patch='{"stringData": { "PREEMPTIVE_FAILOVER_START_TIME": "'"$time_start"'", "PREEMPTIVE_FAILOVER_END_TIME": "'"$time_end"'" }}'
elif [[ "$#" -eq 1 ]]; then

kubectl -n "$namespace" patch secret "$secret_name" \
--patch='{"stringData": { "PREEMPTIVE_FAILOVER_START_TIME": "", "PREEMPTIVE_FAILOVER_END_TIME": "" }}'
fi

# Redeploy the agent.

kubectl -n "$namespace" rollout restart deployment/switch-agent-switchover-agent

0 comments on commit 3d31ef4

Please sign in to comment.