Skip to content

Commit

Permalink
Promote Develop to main for Splunk Operator Release 2.7.1 (#1432)
Browse files Browse the repository at this point in the history
* Use docker-buildx and make smoke tests run

* Add a '.'

* Test again

* Change env variable value

* Trigger int testing

* Minimize changes only to smoke tests to start with

* Initial changes for graviton smoke tests

* Try this

* Add the argument again

* Try passing build arguments

* Add a default value

* Hardcode

* Change tag

* Pull locally

* Don't push for graviton

* Display operator image

* Change eks instance type

* Dump version

* Describe

* Don't need to tag for graviton

* Re-run change kust

* Avoid describe

* Enable everything and try again

* Remove push-latest, re-run pipelines

* Re-run tests

* Enable int tests

* Update error logs

* Further enhance

* Don't use platform in FROM in dockerfile, remove TARGETOSIMAGE, ignore int tests for now

* Trigger int and smoke as well

* Pull image fix - int tests

* Set graviton to true int tests

* Re-trigger

* Trigger

* Re-trigger

* Disable int tests for now.

* Avoid vul testing for graviton for now

* Add support for Ubuntu

* Pass as build arg

* Echo BASE_OS

* Address review comments

* Fix docker builds

* Change logic for Ubuntu

* Test package version

* Run without package versions

* Fix unattended-upgrades

* Build for amd64 as well for pipelines

* Remove space

* Change to AS

* Trigger for 9.2.4 AL2023 ARM

* Try installing certificates on SOK container

* Trigger both arm and ubuntu. Add cert for ubuntu

* Trigger workflows for 9.2.4 AL2023 ARM64

* trigger AL2023 build for splunk 9.2.4

* trigger AL2023 build for splunk 9.3.2

* trigger Ubuntu build for splunk 9.2.4

* trigger Ubuntu build for splunk 9.3.2

* trigger AL2023 build for splunk 9.2.4

* use new label to test app framework tests that hang during teardown

* use shorter label for testing tag

* trigger integration test for PR

* trigger rebuild of sok images for arm64 architectures

* trigger rebuild for sok container on linux arm64

* remove build and test workflow for now

* trigger rebuild for sok container on ubuntu arm64

* separate suite tag for failing test

* get correct standalone for readiness checks

* get correct standalone for readiness checks

* add sleep for managermc1 failing test case

* dump splunk version during consistently check for search head cluster

* clean up new workflows

* merge commit for pulling splunk enterprise image

* re-enable test case, correct merge conflict

* feat: [CSPL-3253]: Change default storageClassName value in PVC

* add back feature branch for integration test workflow trigger

* Initial changes to support deployer spec in SHC CRD

* Integration testing enabled

* Remove SHC updating phase check

* Remove change splunk operator name step in integration test workflow

* Trigger int testing again

* Fix int test bug

* remove specific branch to run integration tests

* Add a comment, rename TC.

* Add UT and return error if not deployer sts

* Add node affinity as well.

* [CSPL-3269] Change error log for clarification (#1422)

* Change error log to info for clarification

* update log statement

* change log back to error, keep new contents

* lowercase error message

* Restore int-test workflow

* CSPL-3156: Add kubectl-splunk Plugin for Executing Splunk Commands within Kubernetes Pods (#1407)

* adding kubectl splunk plugin

Signed-off-by: Vivek Reddy <[email protected]>

* Fix code scanning alert no. 32: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* added auto credentials

Signed-off-by: Vivek Reddy <[email protected]>

* Fix code scanning alert no. 34: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* modularized code with test

Signed-off-by: Vivek Reddy <[email protected]>

* adding workflow for this

Signed-off-by: Vivek Reddy <[email protected]>

* adding workflow for this

Signed-off-by: Vivek Reddy <[email protected]>

* adding name to workflow

Signed-off-by: Vivek Reddy <[email protected]>

* adding name to workflow

Signed-off-by: Vivek Reddy <[email protected]>

* adding name to workflow

Signed-off-by: Vivek Reddy <[email protected]>

* adding branch for test

Signed-off-by: Vivek Reddy <[email protected]>

---------

Signed-off-by: Vivek Reddy <[email protected]>
Co-authored-by: Vivek Reddy <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Arjun Kondur <[email protected]>

* CSPL-2966: Feature: Manual App Updates per Custom Resource (CR) in Splunk Operator (#1395)

* adding cr specific configmap

Signed-off-by: Vivek Reddy <[email protected]>

* formatting changes

* fallback

* adding logic for manual update

Signed-off-by: Vivek Reddy <[email protected]>

* working logic

* commenting unit test for now

* comment unit test for now

* comment unit test in makefile

* fixed field manualUpdate in per Cr config

Signed-off-by: Vivek Reddy <[email protected]>

* add ownership to config

* order changed for config creation

* fix manualUpdate per CR

* unit test cases fixed

* CSPL-2983: doc changes

Signed-off-by: Vivek Reddy <[email protected]>

* fixed unit test cases

Signed-off-by: Vivek Reddy <[email protected]>

* uncomment unit test

Signed-off-by: Vivek Reddy <[email protected]>

* adding go mod changes

Signed-off-by: Vivek Reddy <[email protected]>

* some more changes to go sum

* fixed mc list test case

* fixed searchhead for dev merge

Signed-off-by: Vivek Reddy <[email protected]>

* adding per cr changes

Signed-off-by: Vivek Reddy <[email protected]>

* replaced test label

Signed-off-by: Vivek Reddy <[email protected]>

* added few doc and code changes

Signed-off-by: Vivek Reddy <[email protected]>

---------

Signed-off-by: Vivek Reddy <[email protected]>
Signed-off-by: Vivek Reddy <[email protected]>
Co-authored-by: Vivek Reddy <[email protected]>

* CSPL-3064: Support for Distroless Image Creation in Splunk Operator for Kubernetes (#1421)

* adding support for distoless

Signed-off-by: Vivek Reddy <[email protected]>

* adding document changes for distroless

Signed-off-by: Vivek Reddy <[email protected]>

* adding github workflow for distroless

Signed-off-by: Vivek Reddy <[email protected]>

* adding github workflow for distroless

Signed-off-by: Vivek Reddy <[email protected]>

* adding github workflow for distroless

Signed-off-by: Vivek Reddy <[email protected]>

* adding github workflow for distroless

Signed-off-by: Vivek Reddy <[email protected]>

* Add workflow to push distroless image.

* Add - CSPL-3064 branch to triggers for Arm Distroless Smoke Test WorkFlow

* Add - CSPL-3064 branch to triggers for Arm Distroless Smoke Test WorkFlow

* Add - CSPL-3064 branch to triggers for Arm Distroless Smoke Test WorkFlow

* Remove stuttering from name

* Use correct distroless image name

* Use correct distroless image name

* Comment out vurneability-scan

* Comment out vurneability-scan

* Use correct naming convention in merge develop to main

* Use sidecar in distroless int workflow.

* Update manifest path

* Update sidecar manifest

* Add kustomize patch to deploy sidecar-debug

* Fix kustomize

* Fix command

* Fix sidecar name

* supporting debug pod in pipeline

Signed-off-by: Vivek Reddy <[email protected]>

* renamed sidecar name

* Fix distroless-build-test-push-workflow.yml to not build for amd

* Fix distroless build test push workflow

* fix

* comment vurn scan out

* Update build push distro workflow

* Update distorless-int-test-worfklow

* Review suggestions

* Remove empty lines from install.md

* Remove running on CSPL-3064

---------

Signed-off-by: Vivek Reddy <[email protected]>
Co-authored-by: Vivek Reddy <[email protected]>
Co-authored-by: igor.grzankowski <[email protected]>
Co-authored-by: Igor Grzankowski <[email protected]>

* Splunk Operator 2.7.1 release (#1426)

* [create-pull-request] automated change

* additional changes

* use consistent formatting in ChangeLog.md

* bundle clusterserviceversion updates

---------

Co-authored-by: rlieberman-splunk <[email protected]>
Co-authored-by: rlieberman-splunk <[email protected]>

* Update distroless RC and release tags (#1433)

* update distroless docker image tag

* publish distroless image on releas

---------

Signed-off-by: Vivek Reddy <[email protected]>
Signed-off-by: Vivek Reddy <[email protected]>
Co-authored-by: Arjun Kondur <[email protected]>
Co-authored-by: Arjun Kondur <[email protected]>
Co-authored-by: rlieberman-splunk <[email protected]>
Co-authored-by: Patryk Wasielewski <[email protected]>
Co-authored-by: patrykw-splunk <[email protected]>
Co-authored-by: vivekr-splunk <[email protected]>
Co-authored-by: Vivek Reddy <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: igor.grzankowski <[email protected]>
Co-authored-by: Igor Grzankowski <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: rlieberman-splunk <[email protected]>
  • Loading branch information
12 people authored Feb 3, 2025
1 parent 7d7e2f2 commit 7b323ba
Show file tree
Hide file tree
Showing 86 changed files with 6,348 additions and 755 deletions.
4 changes: 3 additions & 1 deletion .env
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ KUBECTL_VERSION=v1.29.1
AZ_CLI_VERSION=2.30.0
EKSCTL_VERSION=v0.191.0
EKS_CLUSTER_K8_VERSION=1.31
SPLUNK_ENTERPRISE_RELEASE_IMAGE=splunk/splunk:9.3.2
EKS_INSTANCE_TYPE=m5.2xlarge
EKS_INSTANCE_TYPE_ARM64=c6g.4xlarge
SPLUNK_ENTERPRISE_RELEASE_IMAGE=splunk/splunk:9.4.0
303 changes: 303 additions & 0 deletions .github/workflows/arm-AL2023-build-test-push-workflow-AL2023.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,303 @@
name: Arm AL2023 Smoke Test WorkFlow
on:
push:
branches:
- develop
- main
jobs:
check-formating:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Dotenv Action
id: dotenv
uses: falti/dotenv-action@d4d12eaa0e1dd06d5bdc3d7af3bf4c8c93cb5359
- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: ${{ steps.dotenv.outputs.GO_VERSION }}
- name: Check Source formatting
run: make fmt && if [[ $? -ne 0 ]]; then false; fi
- name: Lint source code
run: make vet && if [[ $? -ne 0 ]]; then false; fi
unit-tests:
runs-on: ubuntu-latest
needs: check-formating
steps:
- uses: actions/checkout@v2
- name: Dotenv Action
id: dotenv
uses: falti/dotenv-action@d4d12eaa0e1dd06d5bdc3d7af3bf4c8c93cb5359
- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: ${{ steps.dotenv.outputs.GO_VERSION }}
- name: Install goveralls
run: |
go version
go install github.com/mattn/goveralls@latest
- name: Install Ginkgo
run: |
make setup/ginkgo
go mod tidy
- name: Run Unit Tests
run: make test
- name: Run Code Coverage
run: goveralls -coverprofile=coverage.out -service=circle-ci -repotoken ${{ secrets.COVERALLS_TOKEN }}
- name: Upload Coverage artifacts
uses: actions/[email protected]
with:
name: coverage.out
path: coverage.out
build-operator-image-arm-al2023:
runs-on: ubuntu-latest
needs: unit-tests
env:
SPLUNK_ENTERPRISE_IMAGE: ${{ secrets.SPLUNK_ENTERPRISE_IMAGE }}
SPLUNK_OPERATOR_IMAGE_NAME: splunk/splunk-operator
ECR_REPOSITORY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
steps:
- name: Set up cosign
uses: sigstore/cosign-installer@main

- uses: actions/checkout@v2
- name: Dotenv Action
id: dotenv
uses: falti/dotenv-action@d4d12eaa0e1dd06d5bdc3d7af3bf4c8c93cb5359
- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: ${{ steps.dotenv.outputs.GO_VERSION }}
- name: Install Ginkgo
run: |
make setup/ginkgo
- name: Set up Docker Buildx
uses: docker/[email protected]
- name: Install Operator SDK
run: |
export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac)
export OS=$(uname | awk '{print tolower($0)}')
export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/${{ steps.dotenv.outputs.OPERATOR_SDK_VERSION }}
sudo curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH}
sudo chmod +x operator-sdk_${OS}_${ARCH}
sudo mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_DEFAULT_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Build and push Splunk Operator Image
run: |
export PLATFORMS=linux/arm64,linux/amd64
export BASE_IMAGE=public.ecr.aws/amazonlinux/amazonlinux
export BASE_IMAGE_VERSION=2023
export IMG=${{ secrets.ECR_REPOSITORY }}/${{ env.SPLUNK_OPERATOR_IMAGE_NAME }}:$GITHUB_SHA
make docker-buildx PLATFORMS=$PLATFORMS BASE_IMAGE=$BASE_IMAGE BASE_IMAGE_VERSION=$BASE_IMAGE_VERSION IMG=$IMG
- name: Sign Splunk Operator image with a key
run: |
cosign sign --yes --key env://COSIGN_PRIVATE_KEY ${{ secrets.ECR_REPOSITORY }}/${{ env.SPLUNK_OPERATOR_IMAGE_NAME }}:${{ github.sha }}
env:
COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
vulnerability-scan:
permissions:
actions: read
contents: read
security-events: write
runs-on: ubuntu-latest
needs: build-operator-image-arm-al2023
env:
SPLUNK_ENTERPRISE_IMAGE: ${{ secrets.SPLUNK_ENTERPRISE_IMAGE }}
SPLUNK_OPERATOR_IMAGE_NAME: splunk/splunk-operator
ECR_REPOSITORY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
IMAGE_NAME: ${{ secrets.ECR_REPOSITORY }}/splunk/splunk-operator:${{ github.sha }}
steps:
- name: Set up cosign
uses: sigstore/cosign-installer@main
- uses: actions/checkout@v2
- name: Dotenv Action
id: dotenv
uses: falti/dotenv-action@d4d12eaa0e1dd06d5bdc3d7af3bf4c8c93cb5359
- name: Set up Docker Buildx
uses: docker/[email protected]
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_DEFAULT_REGION }}

- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v1
- name: Pull Splunk Operator Image Locally
run: |
docker pull ${{ env.IMAGE_NAME }}
- name: Verify Signed Splunk Operator image
run: |
cosign verify --key env://COSIGN_PUBLIC_KEY ${{ env.IMAGE_NAME }}
env:
COSIGN_PUBLIC_KEY: ${{ secrets.COSIGN_PUBLIC_KEY }}
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: '${{ env.IMAGE_NAME }}'
format: sarif
#exit-code: 1
severity: 'CRITICAL'
ignore-unfixed: true
output: 'trivy-results.sarif'
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'trivy-results.sarif'
smoke-tests-arm-al2023:
needs: vulnerability-scan
strategy:
fail-fast: false
matrix:
test: [
basic,
appframeworksS1,
managerappframeworkc3,
managerappframeworkm4,
managersecret,
managermc,
]
runs-on: ubuntu-latest
env:
CLUSTER_NODES: 1
CLUSTER_WORKERS: 3
SPLUNK_ENTERPRISE_IMAGE: ${{ secrets.SPLUNK_ENTERPRISE_IMAGE_ARM64 }}
SPLUNK_ENTERPRISE_RELEASE_IMAGE: ${{ secrets.SPLUNK_ENTERPRISE_IMAGE_ARM64 }}
SPLUNK_OPERATOR_IMAGE_NAME: splunk/splunk-operator
SPLUNK_OPERATOR_IMAGE_FILENAME: splunk-operator
TEST_FOCUS: "${{ matrix.test }}"
# This regex matches any string not containing smoke keyword
TEST_TO_SKIP: "^(?:[^s]+|s(?:$|[^m]|m(?:$|[^o]|o(?:$|[^k]|k(?:$|[^e])))))*$"
TEST_CLUSTER_PLATFORM: eks
EKS_VPC_PRIVATE_SUBNET_STRING: ${{ secrets.EKS_VPC_PRIVATE_SUBNET_STRING }}
EKS_VPC_PUBLIC_SUBNET_STRING: ${{ secrets.EKS_VPC_PUBLIC_SUBNET_STRING }}
TEST_BUCKET: ${{ secrets.TEST_BUCKET }}
TEST_INDEXES_S3_BUCKET: ${{ secrets.TEST_INDEXES_S3_BUCKET }}
ECR_REPOSITORY: ${{ secrets.ECR_REPOSITORY }}
PRIVATE_REGISTRY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
ENTERPRISE_LICENSE_LOCATION: ${{ secrets.ENTERPRISE_LICENSE_LOCATION }}
EKS_SSH_PUBLIC_KEY: ${{ secrets.EKS_SSH_PUBLIC_KEY }}
CLUSTER_WIDE: "true"
DEPLOYMENT_TYPE: ""
ARM64: "true"
steps:
- name: Set Test Cluster Name
run: |
echo "TEST_CLUSTER_NAME=eks-integration-test-cluster-${{ matrix.test }}-$GITHUB_RUN_ID" >> $GITHUB_ENV
- name: Chekcout code
uses: actions/checkout@v2
- name: Dotenv Action
id: dotenv
uses: falti/dotenv-action@d4d12eaa0e1dd06d5bdc3d7af3bf4c8c93cb5359
- name: Change splunk enterprise to release image on main branches
if: github.ref == 'refs/heads/main'
run: |
echo "SPLUNK_ENTERPRISE_IMAGE=${{ steps.dotenv.outputs.SPLUNK_ENTERPRISE_RELEASE_IMAGE }}" >> $GITHUB_ENV
- name: Install Kubectl
uses: Azure/setup-kubectl@v3
with:
version: ${{ steps.dotenv.outputs.KUBECTL_VERSION }}
- name: Install Python
uses: actions/setup-python@v2
- name: Install AWS CLI
run: |
curl "${{ steps.dotenv.outputs.AWSCLI_URL}}" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update
aws --version
- name: Setup Go
uses: actions/setup-go@v2
with:
go-version: ${{ steps.dotenv.outputs.GO_VERSION }}
- name: Install Ginkgo
run: |
make setup/ginkgo
- name: Install Helm
run: |
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
DESIRED_VERSION=v3.8.2 bash get_helm.sh
- name: Install EKS CTL
run: |
curl --silent --insecure --location "https://github.com/weaveworks/eksctl/releases/download/${{ steps.dotenv.outputs.EKSCTL_VERSION }}/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version
- name: Set up Docker Buildx
uses: docker/[email protected]
- name: Install Operator SDK
run: |
sudo curl -L -o /usr/local/bin/operator-sdk https://github.com/operator-framework/operator-sdk/releases/download/${{ steps.dotenv.outputs.OPERATOR_SDK_VERSION }}/operator-sdk-${{ steps.dotenv.outputs.OPERATOR_SDK_VERSION }}-x86_64-linux-gnu
sudo chmod +x /usr/local/bin/operator-sdk
- name: Configure Docker Hub credentials
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN}}
- name: Set Splunk Operator image
run: |
echo "SPLUNK_OPERATOR_IMAGE=${{ env.SPLUNK_OPERATOR_IMAGE_NAME }}:$GITHUB_SHA" >> $GITHUB_ENV
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_DEFAULT_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Pull Splunk Enterprise Image
run: docker pull ${{ env.SPLUNK_ENTERPRISE_IMAGE }}
- name: Create EKS cluster
run: |
export EKS_CLUSTER_K8_VERSION=${{ steps.dotenv.outputs.EKS_CLUSTER_K8_VERSION }}
export EKS_INSTANCE_TYPE=${{ steps.dotenv.outputs.EKS_INSTANCE_TYPE_ARM64 }}
make cluster-up
- name: install metric server
run: |
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
- name: install k8s dashboard
run: |
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.5/aio/deploy/recommended.yaml
- name: Setup Kustomize
run: |
sudo snap install kustomize
mkdir -p ./bin
cp /snap/bin/kustomize ./bin/kustomize
- name: Run smoke test
id: smoketest
run: |
make int-test
- name: Collect Test Logs
if: ${{ always() }}
run: |
mkdir -p /tmp/pod_logs
find ./test -name "*.log" -exec cp {} /tmp/pod_logs \;
- name: Archive Pod Logs
if: ${{ always() }}
uses: actions/[email protected]
with:
name: "splunk-pods-logs--artifacts-${{ matrix.test }}"
path: "/tmp/pod_logs/**"
- name: Cleanup Test Case artifacts
if: ${{ always() }}
run: |
make cleanup
make clean
- name: Cleanup up EKS cluster
if: ${{ always() }}
run: |
make cluster-down
Loading

0 comments on commit 7b323ba

Please sign in to comment.