jcasc integration with kubernetes auth not working

[Aransh]

Jenkins and plugins versions report

Environment

Jenkins: 2.414.3
OS: Linux - 5.10.0-21-cloud-amd64
Java: 11.0.20.1 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
ansicolor:1.0.4
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
authentication-tokens:1.53.v1c90fd9191a_b_
aws-credentials:218.v1b_e9466ec5da_
aws-java-sdk:1.12.529-406.vdeff15e5817d
aws-java-sdk-cloudformation:1.12.529-406.vdeff15e5817d
aws-java-sdk-codebuild:1.12.529-406.vdeff15e5817d
aws-java-sdk-ec2:1.12.529-406.vdeff15e5817d
aws-java-sdk-ecr:1.12.529-406.vdeff15e5817d
aws-java-sdk-ecs:1.12.529-406.vdeff15e5817d
aws-java-sdk-efs:1.12.529-406.vdeff15e5817d
aws-java-sdk-elasticbeanstalk:1.12.529-406.vdeff15e5817d
aws-java-sdk-iam:1.12.529-406.vdeff15e5817d
aws-java-sdk-kinesis:1.12.529-406.vdeff15e5817d
aws-java-sdk-logs:1.12.529-406.vdeff15e5817d
aws-java-sdk-minimal:1.12.529-406.vdeff15e5817d
aws-java-sdk-secretsmanager:1.12.529-406.vdeff15e5817d
aws-java-sdk-sns:1.12.529-406.vdeff15e5817d
aws-java-sdk-sqs:1.12.529-406.vdeff15e5817d
aws-java-sdk-ssm:1.12.529-406.vdeff15e5817d
azure-ad:412.vdf45b_6a_b_da_81
azure-cli:0.9
azure-credentials:293.vb_d506148f506
azure-sdk:157.v855da_0b_eb_dc2
blueocean:1.27.8
blueocean-bitbucket-pipeline:1.27.9
blueocean-commons:1.27.9
blueocean-config:1.27.9
blueocean-core-js:1.27.9
blueocean-dashboard:1.27.9
blueocean-display-url:2.4.2
blueocean-events:1.27.9
blueocean-git-pipeline:1.27.9
blueocean-github-pipeline:1.27.9
blueocean-i18n:1.27.9
blueocean-jwt:1.27.9
blueocean-personalization:1.27.9
blueocean-pipeline-api-impl:1.27.9
blueocean-pipeline-editor:1.27.9
blueocean-pipeline-scm-api:1.27.9
blueocean-rest:1.27.9
blueocean-rest-impl:1.27.9
blueocean-web:1.27.9
bootstrap5-api:5.3.2-2
bouncycastle-api:2.29
branch-api:2.1128.v717130d4f816
build-name-setter:2.4.0
build-user-vars-plugin:1.9
buildtriggerbadge:251.vdf6ef853f3f5
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.0.2
cloudbees-bitbucket-branch-source:848.v42c6a_317eda_e
cloudbees-disk-usage-simple:187.v6378d330d1d4
cloudbees-folder:6.858.v898218f3609d
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-94.v3e1f4a_926e49
configuration-as-code:1714.v09593e830cfa
credentials:1307.v3757c78f17c3
credentials-binding:642.v737c34dea_6c2
custom-tools-plugin:0.8
data-tables-api:1.13.6-5
display-url-api:2.200.vb_9327d658781
durable-task:523.va_a_22cf15d5e0
echarts-api:5.4.0-7
email-ext:2.102
extended-choice-parameter:376.v2e02857547b_a_
favorite:2.4.3
font-awesome-api:6.4.2-1
generic-webhook-trigger:1.88.0
git:5.2.0
git-client:4.5.0
github:1.37.3.1
github-api:1.316-451.v15738eef3414
github-branch-source:1741.va_3028eb_9fd21
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
hashicorp-vault-plugin:361.v44fea_4fc08d9
htmlpublisher:1.32
instance-identity:185.v303dc7c645f9
ionicons-api:56.v1b_1c8c49374e
jackson2-api:2.15.3-366.vfe8d1fa_f8c87
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jenkins-design-language:1.27.9
jjwt-api:0.11.5-77.v646c772fddb_0
job-dsl:1.87
jquery3-api:3.7.1-1
junit:1240.vf9529b_881428
kubernetes:4054.v2da_8e2794884
kubernetes-client-api:6.8.1-224.vd388fca_4db_3b_
kubernetes-credentials:0.11
lockable-resources:1185.v0c528656ce04
mailer:463.vedf8358e006b_
matrix-auth:3.2.1
matrix-project:818.v7eb_e657db_924
metrics:4.2.18-442.v02e107157925
mina-sshd-api-common:2.11.0-86.v836f585d47fa_
mina-sshd-api-core:2.11.0-86.v836f585d47fa_
multibranch-scan-webhook-trigger:1.0.9
okhttp-api:4.11.0-157.v6852a_a_fa_ec11
pipeline-aws:1.43
pipeline-build-step:516.v8ee60a_81c5b_9
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:689.veec561a_dee13
pipeline-input-step:477.v339683a_8d55e
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2150.v4cfd8916915c
pipeline-model-definition:2.2150.v4cfd8916915c
pipeline-model-extensions:2.2150.v4cfd8916915c
pipeline-rest-api:2.34
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2150.v4cfd8916915c
pipeline-stage-view:2.33
pipeline-utility-steps:2.16.0
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:3.6.0
prism-api:1.29.0-8
prometheus:2.3.3
pubsub-light:1.18
scm-api:676.v886669a_199a_a_
script-security:1275.v23895f409fb_d
snakeyaml-api:2.2-111.vc6598e30cc65
sse-gateway:1.26
ssh-credentials:308.ve4497b_ccd8f4
sshd:3.312.v1c601b_c83b_0e
structs:325.vcb_307d2a_2782
terraform:1.0.10
timestamper:1.26
token-macro:384.vf35b_f26814ec
trilead-api:2.84.v72119de229b_7
uno-choice:2.8.0
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1283.v99c10937efcb_
workflow-basic-steps:1042.ve7b_140c4a_e0c
workflow-cps:3806.va_3a_6988277b_2
workflow-durable-task-step:1289.v4d3e7b_01546b_
workflow-job:1360.vc6700e3136f5
workflow-multibranch:756.v891d88f2cd46
workflow-scm-step:415.v434365564324
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:865.v43e78cc44e0d

What Operating System are you using (both controller, and any agents involved in the problem)?

Running on k8s, Linux based

Reproduction steps

1. Install Jenkins on a kubernetes cluster using the Jenkins Helm chart
2. Install the "HashiCorp Vault" plugin
3. Configure kubernetes authentication between the Jenkins cluster and Hashicorp Vault
4. Add required CASC env vars as described in documentation:

• CASC_VAULT_URL="https://" (url of the vault server)
• CASC_VAULT_KUBERNETES_ROLE="jenkins-role" (name of role in Vault)
• CASC_VAULT_MOUNT="csi_lke_jenkins" (name of said cluster auth mount in vault)
• CASC_VAULT_PATHS="csi_lke_jenkins/sso-secrets" (path of an example secret)

5. Update jcasc file to take values from vault path, as described in documentation, example (tried using 3 different approaches to extract secrets under path csi_lke_jenkins/sso-secrets):

      azure:
        tenant: ${JENKINS_AUTH_AZUREAD_TENANT_ID}
        clientId: ${csi_lke_jenkins /sso-credentials/JENKINS_AUTH_AZUREAD_CLIENT_ID}
        clientSecret: ${sso-credentials/JENKINS_AUTH_AZUREAD_CLIENT_SECRET}

6. Jenkins config refreshes, none of the configured values are there...

Expected Results

I expected either the values from vault will be inputted in the configuration file,
or at least to get some kind of error log specifying why this isn't working, whether anything failed

Actual Results

Values are simply not added to the jcasc config file, not seeing any logs from the plugin

Anything else?

Important to note this is definitely not an issue with the kubernetes authentication configuration, as we use the same service account and role for the general vault integration, and it is working as intended.
I am also not sure if the issue is with kubernetes auth support or generally with the jcasc integration, as the kubernetes auth is the only method I am currently able to test.

[Aransh]

Also tagging @netfalo @marcoreni and @jetersen Who I see were involved in the implementation of this in #66 #52