Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: GitHub action for secrets e2e test #3682

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a4a12af
ADD: initial AWS testing resources CDK
chenqi0805 Oct 13, 2023
23e63d1
MNT: renaming
chenqi0805 Oct 23, 2023
3a6d4ad
E2E: basicLogWithAwsSecretsEndToEndTest
chenqi0805 Nov 14, 2023
f1e2e69
MAINT: refactoring on build.gradle
chenqi0805 Nov 14, 2023
fdce0f1
MAINT: merge main and resolve conflict
chenqi0805 Nov 14, 2023
98beb6b
Merge branch 'main' into e2e/aws-secrets-tests
chenqi0805 Nov 15, 2023
40a9025
MAINT: use AWS_PROFILE
chenqi0805 Nov 15, 2023
520d68b
MAINT: check aws credentials
chenqi0805 Nov 16, 2023
175bad3
maint: check absolute path of .aws
chenqi0805 Nov 16, 2023
b8ef85c
MAINT: check output
chenqi0805 Nov 16, 2023
57c794f
MAINT: check output
chenqi0805 Nov 16, 2023
bfd9bd6
MAINT: check step output
chenqi0805 Nov 16, 2023
caab290
MAINT: aws configure
chenqi0805 Nov 16, 2023
bbbe5c0
MAINT: set files
chenqi0805 Nov 16, 2023
a9c6dd7
MNT: format
chenqi0805 Nov 16, 2023
55d632e
MAINT: check files
chenqi0805 Nov 16, 2023
7df6949
testing
chenqi0805 Nov 16, 2023
cab4fc0
MAINT: run actual tests
chenqi0805 Nov 16, 2023
9c871d4
Merge branch 'main' into maint/github-action-for-e2e-secrets
chenqi0805 Nov 16, 2023
5e9b66e
TST: assume new role
chenqi0805 Nov 16, 2023
a0edad4
MAINT: add test
chenqi0805 Nov 16, 2023
478e414
MAINT: title
chenqi0805 Nov 16, 2023
ea75910
MAINT: back test on branch
chenqi0805 Nov 16, 2023
27eff48
MAINT: revert
chenqi0805 Nov 16, 2023
9df0981
MAINT: job name
chenqi0805 Nov 16, 2023
b5c36ec
MAINT: file name
chenqi0805 Nov 16, 2023
421aa69
Merge branch 'main' into maint/github-action-for-e2e-secrets
chenqi0805 Nov 17, 2023
b455902
MAINT: us-east-2
chenqi0805 Nov 17, 2023
e9039cf
MAINT: manual workflow trigger
chenqi0805 Nov 21, 2023
7d0a098
MAINT: add back variable
chenqi0805 Nov 27, 2023
25e8686
MAINT: use repo variables
chenqi0805 Nov 27, 2023
9e737df
Merge branch 'main' into maint/github-action-for-e2e-secrets
chenqi0805 Nov 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/data-prepper-aws-secrets-e2e-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# This workflow will build a Java project with Gradle
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-gradle

name: Data Prepper AWS secrets End-to-end test with Gradle
on:
push:
branches: [ main ]
pull_request:
workflow_dispatch:
# permission can be added at job level or workflow level
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout

jobs:
build:
strategy:
matrix:
java: [ 11, 17, 21, docker ]
test: [ 'basicLogWithAwsSecretsEndToEndTest' ]
fail-fast: false

runs-on: ubuntu-latest
steps:
- name: Git clone the repository
uses: actions/checkout@v4
- name: configure aws credentials
id: creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.TEST_IAM_ROLE_ARN }}
aws-region: ${{ vars.TEST_REGION }}
output-credentials: true
- name: get caller identity 1
run: |
aws sts get-caller-identity
- name: Configure AWS Credentials file
run: |
aws configure set default.region ${{ vars.TEST_REGION }}
aws configure set default.aws_access_key_id ${{ steps.creds.outputs.aws-access-key-id }}
aws configure set default.aws_secret_access_key ${{ steps.creds.outputs.aws-secret-access-key }}
aws configure set default.aws_session_token ${{ steps.creds.outputs.aws-session-token }}
- name: Set up JDK 11
uses: actions/setup-java@v1
with:
java-version: 11
- name: Checkout Data Prepper
uses: actions/checkout@v2
- name: Run basic grok with AWS secrets end-to-end tests with Gradle
run: AWS_PROFILE=default ./gradlew -PendToEndJavaVersion=${{ matrix.java }} :e2e-test:log:${{ matrix.test }}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ pipeline_configurations:
secrets:
opensearch-sink:
secret_id: "opensearch-sink-basic-credentials"
region: "us-east-2"
grok-pipeline:
source:
http:
Expand Down
Loading