Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: backend oracle extract and load #276

Merged
merged 52 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
f599472
feat: backend el initial commit
mishraomp Jan 23, 2025
7955019
chore: intermediate commit
mishraomp Jan 24, 2025
7d03d48
chore: intermediate commit switching to spring boot from quarkus
mishraomp Jan 25, 2025
6d3d0b7
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 25, 2025
789fc77
fix: trivy
mishraomp Jan 25, 2025
ddff120
chore: some more entities and more csv processing
mishraomp Jan 26, 2025
778ceff
chore: more code reuse
mishraomp Jan 26, 2025
a9cad89
chore: remove controller will be added later if needed currently run …
mishraomp Jan 26, 2025
60fa78f
chore: some more entities added
mishraomp Jan 26, 2025
a8b7f10
chore: more entities including attachment
mishraomp Jan 27, 2025
018e27b
chore: added more entities
mishraomp Jan 27, 2025
1a91c2f
fix: compilation error
mishraomp Jan 27, 2025
6976cce
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 27, 2025
f18c253
feat: adding backend el build deploy for openshift
mishraomp Jan 27, 2025
8d0ba81
chore: ignore helm charts for precommit
mishraomp Jan 27, 2025
5ef076d
chore: precommit I hope I made it this time
mishraomp Jan 27, 2025
dd9db33
Merge branch 'feat/oracle-el-gha-ci-cd' of https://github.com/bcgov/n…
mishraomp Jan 28, 2025
58febf8
chore: more logic for attachment processing, fix dockerfile
mishraomp Jan 28, 2025
7a926dc
Merge branch 'feat/oracle-el-api' of https://github.com/bcgov/nr-rec-…
mishraomp Jan 28, 2025
70b8e04
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 28, 2025
b332bf5
chore: support openshift deployment, dev and prod, configure aws secr…
mishraomp Jan 28, 2025
bfffc6c
fix: build file context
mishraomp Jan 28, 2025
7af8f63
fix: build context
mishraomp Jan 28, 2025
a6d8707
fix: helm deploy add missing parameter name
mishraomp Jan 28, 2025
c561456
fix: cronjob yaml indent
mishraomp Jan 28, 2025
8c42f55
chore: add missing env var and update job names
mishraomp Jan 28, 2025
2f26ae7
chore: add missing env var
mishraomp Jan 28, 2025
3844679
chore: make JDB URL env var
mishraomp Jan 28, 2025
143dab3
chore: make shell script to debug issues
mishraomp Jan 28, 2025
18e31b0
chore: try running without init, init will be deleted later
mishraomp Jan 28, 2025
b47cf14
fix: Dockerfile
mishraomp Jan 28, 2025
1cbb806
fix: wrap values with quotes
mishraomp Jan 28, 2025
66eb2d2
chore: remove init container and related charts, add certs to the mai…
mishraomp Jan 28, 2025
ef201be
fix: helm remove cronjob cert pvc
mishraomp Jan 29, 2025
2d33859
chore: add s3 sync between aws and openshift
mishraomp Jan 29, 2025
adf5034
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 29, 2025
0e43a89
fix: precommit
mishraomp Jan 29, 2025
551b94b
Merge branch 'feat/oracle-el-api' of https://github.com/bcgov/nr-rec-…
mishraomp Jan 29, 2025
4c38d47
chore: add workflow dispatch to trigger manually when needed
mishraomp Jan 29, 2025
14265ea
fix: java start command to reduce noise and added some more jvm args
mishraomp Jan 29, 2025
2da0c84
docs: add documentation
mishraomp Jan 29, 2025
3860ecb
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 30, 2025
6c7226f
chore: align with schema, column ordering and remove subpath
mishraomp Jan 30, 2025
1395c49
fix: tf code to not error during tf plan in pr
mishraomp Jan 30, 2025
da6a1cd
fix: tf and gha
mishraomp Jan 30, 2025
5116325
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 30, 2025
381c1b9
docs: OpenShift deployment
mishraomp Jan 30, 2025
5aa5507
Merge branch 'feat/oracle-el-api' of https://github.com/bcgov/nr-rec-…
mishraomp Jan 30, 2025
6c4602e
chore: update s3 location
mishraomp Jan 30, 2025
1445d80
fix: entity activity rank
mishraomp Jan 30, 2025
c2e901e
fix: entity align with db and add missing columns
mishraomp Jan 30, 2025
d4019a9
Merge branch 'main' into feat/oracle-el-api
mishraomp Jan 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/openshift-oracle-s3-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: OpenShit Oracle S3 Sync
on:
schedule:
- cron: '0 0 * * *'
permissions:
id-token: write # This is required for requesting the JWT
contents: write # This is required for actions/checkout
packages: write
env:
AWS_REGION: ca-central-1
jobs:
read-s3-credentials-from-aws:
runs-on: ubuntu-24.04
environment: dev
DerekRoberts marked this conversation as resolved.
Show resolved Hide resolved
outputs:
AWS_ACCESS_KEY_ID: ${{ steps.read-s3-credentials.outputs.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ steps.read-s3-credentials.outputs.AWS_SECRET_ACCESS_KEY }}
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
role-session-name: openshift-oracle-s3-sync
aws-region: ${{ env.AWS_REGION }}
- name: Read S3 Credentials
id: read-s3-credentials
shell: bash
run: |
AWS_CREDENTIALS=$(aws ssm get-parameter \
--name "/iam_users/fta-rst-el-s3-upload-service-account_keys" \
--with-decryption \
--query "Parameter.Value" \
--output text)
# Extract and mask credentials
ACCESS_KEY=$(echo $AWS_CREDENTIALS | jq -r '.current.AccessKeyID')
SECRET_KEY=$(echo $AWS_CREDENTIALS | jq -r '.current.SecretAccessKey')
# Mask values in logs
echo "::add-mask::$ACCESS_KEY"
echo "::add-mask::$SECRET_KEY"
# Set outputs
echo "AWS_ACCESS_KEY_ID=$ACCESS_KEY" >> $GITHUB_OUTPUT
echo "AWS_SECRET_ACCESS_KEY=$SECRET_KEY" >> $GITHUB_OUTPUT

update-openshift-secret:
runs-on: ubuntu-24.04
needs: read-s3-credentials-from-aws
environment: openshift-dev
steps:
- name: Update OpenShift Secret
uses: bcgov/[email protected]
env:
DB_PASSWORD: ${{ secrets.dbPassword }} # handle special characters.
with:
oc_namespace: ${{ secrets.oc_namespace }}
oc_token: ${{ secrets.oc_token }}
oc_server: ${{ vars.oc_server }}
commands: |
oc delete secret aws-secrets || true
oc create secret generic aws-secrets \
--from-literal=AWS_ACCESS_KEY="${{ needs.read-s3-credentials-from-aws.outputs.AWS_ACCESS_KEY_ID }}" \
--from-literal=AWS_SECRET_KEY="${{ needs.read-s3-credentials-from-aws.outputs.AWS_SECRET_ACCESS_KEY }}"
1 change: 0 additions & 1 deletion backend-el/run_app.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,3 @@ if [ -n "${SSL_DEBUG}" ]; then
else
java -Duser.name=REC-ORACLE-EL -Xms512m -Xmx512m -jar app.jar
fi
``
Loading