Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler Longhaul Tests #238

Merged
merged 15 commits into from
Nov 20, 2024
Merged

Scheduler Longhaul Tests #238

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
e2e905a
robust jobs api longhaul test
cicoyle Oct 21, 2024
c6bd947
fix file name
cicoyle Oct 21, 2024
439192e
add actor reminder test apps: client + server
cicoyle Oct 25, 2024
ab84339
update ports and add dockerfile
cicoyle Oct 25, 2024
2cd4dd6
add workflow app
cicoyle Oct 30, 2024
07aaad2
add bicep per app
cicoyle Oct 30, 2024
ac21c48
add workflow deploy yaml updates
cicoyle Oct 30, 2024
9e2fde4
add redeploy config and aks/app bicep files
cicoyle Oct 30, 2024
16c161c
tweak compose
cicoyle Nov 6, 2024
061cc23
update ctx + cleanup + readme
cicoyle Nov 19, 2024
c4a4a42
wf test updates
cicoyle Nov 19, 2024
342f5a1
update actor reminder client code
cicoyle Nov 19, 2024
d5168c2
Update scheduler-actor-reminders/Dockerfile-client
cicoyle Nov 20, 2024
eb5cab9
fix ports and infra config
cicoyle Nov 20, 2024
1e5877a
fix docker-compose image name
cicoyle Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/dapr-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,7 @@ jobs:
kubectl apply -f ./longhaul-test/pubsub-workflow-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/pubsub-workflow-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/snapshot-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/snapshot-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/validation-worker-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/validation-worker-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/workflow-gen-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/workflow-gen-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/scheduler-jobs-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-jobs-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/scheduler-actor-reminders-server-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-actor-reminders-server-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/scheduler-actor-reminders-client-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-actor-reminders-client-app -n ${{ env.APP_NAMESPACE }}
cicoyle marked this conversation as resolved.
Show resolved Hide resolved
kubectl apply -f ./longhaul-test/scheduler-workflow-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-workflow-app -n ${{ env.APP_NAMESPACE }}
5 changes: 4 additions & 1 deletion .github/workflows/dapr-longhaul-weekly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,7 @@ jobs:
kubectl apply -f ./longhaul-test/snapshot-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/snapshot-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/validation-worker-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/validation-worker-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/workflow-gen-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/workflow-gen-app -n ${{ env.APP_NAMESPACE }}

kubectl apply -f ./longhaul-test/scheduler-jobs-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-jobs-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/scheduler-actor-reminders-server-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-actor-reminders-server-app -n ${{ env.APP_NAMESPACE }}
kubectl apply -f ./longhaul-test/scheduler-actor-reminders-client-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-actor-reminders-client-app -n ${{ env.APP_NAMESPACE }}
cicoyle marked this conversation as resolved.
Show resolved Hide resolved
kubectl apply -f ./longhaul-test/scheduler-workflow-deploy.yml -n ${{ env.APP_NAMESPACE }} && kubectl rollout restart deploy/scheduler-workflow-app -n ${{ env.APP_NAMESPACE }}
84 changes: 84 additions & 0 deletions .github/workflows/scheduler-actor-reminders-client-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ------------------------------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------------------------------

name: build-scheduler-actor-reminders-client

on:
push:
branches:
- master
paths:
- 'scheduler-actor-reminders/client/**'
- '.github/workflows/scheduler-actor-reminders-client-build.yml'
- '.github/workflows/dapr-deploy.yml'
pull_request:
branches:
- master
paths:
- 'scheduler-actor-reminders/client/**'

jobs:
build:
name: build scheduler-actor-reminders-client
runs-on: ubuntu-latest
env:
APP_REGISTRY: daprtests.azurecr.io
APP_IMAGE_NAME: scheduler-actor-reminders-client
# TODO: APP_VER needs to be versioned correctly
APP_VER: dev
APP_DIR: ./scheduler-actor-reminders-client
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: Check out code
uses: actions/checkout@v2
- name: docker login
if: github.event_name != 'pull_request'
uses: docker/login-action@v1
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_REGISTRY_ID }}
password: ${{ secrets.DOCKER_REGISTRY_PASS }}
- name: Build scheduler-actor-reminders-client app docker image
run: |
docker compose build ${{ env.APP_IMAGE_NAME }}
docker tag ${{ env.APP_IMAGE_NAME }} ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Push scheduler-actor-reminders-client app image to dockerhub
if: github.event_name != 'pull_request'
run: |
docker push ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Copy deployment yaml to archieve
run: |
mkdir -p ${{ env.ARTIFACT_DIR }}
cp ./longhaul-test/*.yml ${{ env.ARTIFACT_DIR }}
- name: Upload artifacts
uses: actions/upload-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
deploy:
name: deploy scheduler-actor-reminders-client to test cluster
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
env:
APP_NAMESPACE: longhaul-test
TEST_CLUSTER_NAME: aks-longhaul-release
TEST_RESOURCE_GROUP: aks-longhaul-release
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: download artifacts
uses: actions/download-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
- name: Login Azure
run: |
az login --service-principal -u ${{ secrets.AZURE_LOGIN_USER }} -p ${{ secrets.AZURE_LOGIN_PASS }} --tenant ${{ secrets.AZURE_TENANT }} --output none
- name: Set up kubeconf for longhaul test environment
run: |
az aks get-credentials -n ${{ env.TEST_CLUSTER_NAME }} -g ${{ env.TEST_RESOURCE_GROUP }}
- name: Deploy apps to longhaul test environment
run: |
kubectl apply -n ${{ env.APP_NAMESPACE }} -f ${{ env.ARTIFACT_DIR }}/scheduler-actor-reminders-client-deploy.yml & kubectl rollout restart -n ${{ env.APP_NAMESPACE }} deploy/scheduler-actor-reminders-client-app
84 changes: 84 additions & 0 deletions .github/workflows/scheduler-actor-reminders-server-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ------------------------------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------------------------------

name: build-scheduler-actor-reminders-server

on:
push:
branches:
- master
paths:
- 'scheduler-actor-reminders/server/**'
- '.github/workflows/scheduler-actor-reminders-server-build.yml'
- '.github/workflows/dapr-deploy.yml'
pull_request:
branches:
- master
paths:
- 'scheduler-actor-reminders/server/**'

jobs:
build:
name: build scheduler-actor-reminders-server
runs-on: ubuntu-latest
env:
APP_REGISTRY: daprtests.azurecr.io
APP_IMAGE_NAME: scheduler-actor-reminders-server
# TODO: APP_VER needs to be versioned correctly
APP_VER: dev
APP_DIR: ./scheduler-actor-reminders-server
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: Check out code
uses: actions/checkout@v2
- name: docker login
if: github.event_name != 'pull_request'
uses: docker/login-action@v1
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_REGISTRY_ID }}
password: ${{ secrets.DOCKER_REGISTRY_PASS }}
- name: Build scheduler-actor-reminders-server app docker image
run: |
docker compose build ${{ env.APP_IMAGE_NAME }}
docker tag ${{ env.APP_IMAGE_NAME }} ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Push scheduler-actor-reminders-server app image to dockerhub
if: github.event_name != 'pull_request'
run: |
docker push ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Copy deployment yaml to archieve
run: |
mkdir -p ${{ env.ARTIFACT_DIR }}
cp ./longhaul-test/*.yml ${{ env.ARTIFACT_DIR }}
- name: Upload artifacts
uses: actions/upload-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
deploy:
name: deploy scheduler-actor-reminders-server to test cluster
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
env:
APP_NAMESPACE: longhaul-test
TEST_CLUSTER_NAME: aks-longhaul-release
TEST_RESOURCE_GROUP: aks-longhaul-release
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: download artifacts
uses: actions/download-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
- name: Login Azure
run: |
az login --service-principal -u ${{ secrets.AZURE_LOGIN_USER }} -p ${{ secrets.AZURE_LOGIN_PASS }} --tenant ${{ secrets.AZURE_TENANT }} --output none
- name: Set up kubeconf for longhaul test environment
run: |
az aks get-credentials -n ${{ env.TEST_CLUSTER_NAME }} -g ${{ env.TEST_RESOURCE_GROUP }}
- name: Deploy apps to longhaul test environment
run: |
kubectl apply -n ${{ env.APP_NAMESPACE }} -f ${{ env.ARTIFACT_DIR }}/scheduler-actor-reminders-server-deploy.yml & kubectl rollout restart -n ${{ env.APP_NAMESPACE }} deploy/scheduler-actor-reminders-server-app
84 changes: 84 additions & 0 deletions .github/workflows/scheduler-jobs-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ------------------------------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------------------------------

name: build-scheduler-jobs

on:
push:
branches:
- master
paths:
- 'scheduler-jobs/**'
- '.github/workflows/scheduler-jobs-build.yml'
- '.github/workflows/dapr-deploy.yml'
pull_request:
branches:
- master
paths:
- 'scheduler-jobs/**'

jobs:
build:
name: build scheduler-jobs
runs-on: ubuntu-latest
env:
APP_REGISTRY: daprtests.azurecr.io
APP_IMAGE_NAME: scheduler-jobs
# TODO: APP_VER needs to be versioned correctly
APP_VER: dev
APP_DIR: ./scheduler-jobs
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: Check out code
uses: actions/checkout@v2
- name: docker login
if: github.event_name != 'pull_request'
uses: docker/login-action@v1
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_REGISTRY_ID }}
password: ${{ secrets.DOCKER_REGISTRY_PASS }}
- name: Build scheduler-jobs app docker image
run: |
docker compose build ${{ env.APP_IMAGE_NAME }}
docker tag ${{ env.APP_IMAGE_NAME }} ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Push scheduler-jobs app image to dockerhub
if: github.event_name != 'pull_request'
run: |
docker push ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Copy deployment yaml to archieve
run: |
mkdir -p ${{ env.ARTIFACT_DIR }}
cp ./longhaul-test/*.yml ${{ env.ARTIFACT_DIR }}
- name: Upload artifacts
uses: actions/upload-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
deploy:
name: deploy scheduler-jobs to test cluster
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
env:
APP_NAMESPACE: longhaul-test
TEST_CLUSTER_NAME: aks-longhaul-release
TEST_RESOURCE_GROUP: aks-longhaul-release
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: download artifacts
uses: actions/download-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
- name: Login Azure
run: |
az login --service-principal -u ${{ secrets.AZURE_LOGIN_USER }} -p ${{ secrets.AZURE_LOGIN_PASS }} --tenant ${{ secrets.AZURE_TENANT }} --output none
- name: Set up kubeconf for longhaul test environment
run: |
az aks get-credentials -n ${{ env.TEST_CLUSTER_NAME }} -g ${{ env.TEST_RESOURCE_GROUP }}
- name: Deploy apps to longhaul test environment
run: |
kubectl apply -n ${{ env.APP_NAMESPACE }} -f ${{ env.ARTIFACT_DIR }}/scheduler-jobs-deploy.yml & kubectl rollout restart -n ${{ env.APP_NAMESPACE }} deploy/scheduler-jobs-app
84 changes: 84 additions & 0 deletions .github/workflows/scheduler-workflow-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ------------------------------------------------------------
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# ------------------------------------------------------------

name: build-scheduler-workflow

on:
push:
branches:
- master
paths:
- 'scheduler-workflow/**'
- '.github/workflows/scheduler-workflow-build.yml'
- '.github/workflows/dapr-deploy.yml'
pull_request:
branches:
- master
paths:
- 'scheduler-workflow/**'

jobs:
build:
name: build scheduler-workflow
runs-on: ubuntu-latest
env:
APP_REGISTRY: daprtests.azurecr.io
APP_IMAGE_NAME: scheduler-workflow
# TODO: APP_VER needs to be versioned correctly
APP_VER: dev
APP_DIR: ./scheduler-workflow
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: Check out code
uses: actions/checkout@v2
- name: docker login
if: github.event_name != 'pull_request'
uses: docker/login-action@v1
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_REGISTRY_ID }}
password: ${{ secrets.DOCKER_REGISTRY_PASS }}
- name: Build scheduler-workflow app docker image
run: |
docker compose build ${{ env.APP_IMAGE_NAME }}
docker tag ${{ env.APP_IMAGE_NAME }} ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Push scheduler-workflow app image to dockerhub
if: github.event_name != 'pull_request'
run: |
docker push ${{ env.APP_REGISTRY }}/${{ env.APP_IMAGE_NAME }}:${{ env.APP_VER }}
- name: Copy deployment yaml to archieve
run: |
mkdir -p ${{ env.ARTIFACT_DIR }}
cp ./longhaul-test/*.yml ${{ env.ARTIFACT_DIR }}
- name: Upload artifacts
uses: actions/upload-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
deploy:
name: deploy scheduler-workflow to test cluster
needs: build
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
env:
APP_NAMESPACE: longhaul-test
TEST_CLUSTER_NAME: aks-longhaul-release
TEST_RESOURCE_GROUP: aks-longhaul-release
ARTIFACT_DIR: ./deploy_artifact
steps:
- name: download artifacts
uses: actions/download-artifact@master
with:
name: longhaul-test
path: ${{ env.ARTIFACT_DIR }}
- name: Login Azure
run: |
az login --service-principal -u ${{ secrets.AZURE_LOGIN_USER }} -p ${{ secrets.AZURE_LOGIN_PASS }} --tenant ${{ secrets.AZURE_TENANT }} --output none
- name: Set up kubeconf for longhaul test environment
run: |
az aks get-credentials -n ${{ env.TEST_CLUSTER_NAME }} -g ${{ env.TEST_RESOURCE_GROUP }}
- name: Deploy apps to longhaul test environment
run: |
kubectl apply -n ${{ env.APP_NAMESPACE }} -f ${{ env.ARTIFACT_DIR }}/scheduler-workflow-deploy.yml & kubectl rollout restart -n ${{ env.APP_NAMESPACE }} deploy/scheduler-workflow-app
Loading
Loading