Enhancement Request: azurerm backend authentication upgrade to match provider

[jaredfholgate]

Terraform Version

1.6.5

Use Cases

I want to use az CLI service principal authentication in my CI / CD pipelines. I am able to do that with the upgraded authentication in the provider for plan and apply, but init does not support it for the azurerm backend.

I'd like to have a consistent authentication experience across all commands.

The original driver for this is running terraform test with OIDC auth. Since the Azure DevOps ID Token is only valid for 10 minutes and each test configures a new provider for each time it times out if there are a few tests in there.

Attempted Solutions

There is no solution other than using the existing mechanism to supply service principal credentials.

Proposal

Update the azurerm backend authentication code
to match the provider code.

References

For clarity, this is the line in v0.43.0 of the target library that explicitly breaks SP auth with CLI: https://github.com/hashicorp/go-azure-helpers/blob/202cb910fbce8b8a78fe6e40177569b6ad59e885/authentication/auth_method_azure_cli_token.go#L72

The target library has significantly changed since that version and that code no longer exists in there.

No response

[crw]

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

[Noel-Jones]

Is there likely to be a resolution for this? I've ended up here having spent a fair while trying to reasearch why my CI/CD pipelines fail to apply changes if I don't complete the manual validation within 10 minutes. After this time applying the plan file fails with the error "AADSTS700024: Client assertion is not within its valid time range". All this became an issue after I switched to the "recommended" use of Workload Identites. I arrived here via microsoft/azure-pipelines-terraform#201 and microsoft/azure-pipelines-terraform#89

It does not appear to be clear in the initial use case / description just how much of an issue this appears to be for many people. 33 people have upvoted this but I believe a lot more would if they found their way here.

I have seen a few references to workarounds but with no clear examples of quite what to do. If anyone can advise a workaround then I would be grateful if they could post details.

[crw]

@Noel-Jones it is possible this could be addressed, but difficult to say how likely or when. Thanks for the additional use case!

[magodo]

Related: #35381 (though closed)

[jaredfholgate]

I think it is worth re-visiting this and adding some clarity as there are a few overlapping issues at play here:

• The original intent for this issue was to add support for az cli authentication with a service principal to bring it inline with the providers. This would also have provided a workaround for the token expiry issue people are facing, albeit not a native solution to the problem.
• With regards to the token expiry issue in Azure DevOps, that could now be solved using the native capability in the Go SDK: https://devblogs.microsoft.com/azure-sdk/improve-security-posture-in-azure-service-connections-with-azurepipelinescredential/. This probably warrants a separate issue, but fundamentally the authentication library needs to be updated. Cross referencing this issue for azapi: azapi authentication failed when resource depends on a terraform creation that takes more than 10 minutes Azure/terraform-provider-azapi#515 (comment)
• The other thing people may come here for is support for using different identities for provider and backend auth. This could be achieved by adding alternative environment variables that don't overlap with the provider environment variables.

Hopefully that helps in some way or I may have caused extra confusion...

[jpomfret]

Hey folks,
Just here to ask for any updates on this, or if there is a suggested workaround - I've had a read through some issues but it's not clear whether there is a suitable workaround.

I am in the same situation as @Noel-Jones - in that I have a pipeline that does init, plan, then a manual approval step - then the apply stage is next - if the gap between init and apply is > 10 mins I get the following error:

AADSTS700024: Client assertion is not within its valid time range. 

I'm using TerraformCLI@1 tasks - would changing to use AzureCLI@2 tasks work here? or is the problem not in the task, but in the terraform calls themselves?

Cheers

[gbordier]

Hey folks, it looks like we had a solution for this in #35381 that conflicted with a change @manicminer wanted to do on the azure backend. do we have any more news on this ?

[Nullh]

@jpomfret Did you have a go with AzureCli@2 task? I've got a similar issue and unpicking all the submodules to give it a go is a bit daunting if it doesn't help :D

[jaredfholgate]

Hey folks, Just here to ask for any updates on this, or if there is a suggested workaround - I've had a read through some issues but it's not clear whether there is a suitable workaround.

I am in the same situation as @Noel-Jones - in that I have a pipeline that does init, plan, then a manual approval step - then the apply stage is next - if the gap between init and apply is > 10 mins I get the following error:

AADSTS700024: Client assertion is not within its valid time range. 

I'm using TerraformCLI@1 tasks - would changing to use AzureCLI@2 tasks work here? or is the problem not in the task, but in the terraform calls themselves?

Cheers

Hi @jpomfret . Did you resolve this? Are you supplying the id token to the init via CLI args? If so, don't do that. Supply it via an env var instead. If you supply it via cli args it gets cached in the plan file and will result in the timeout you are seeing.

[jpomfret]

Hi @jaredfholgate - yes I am passing it in as a CLI arg.
To use an env var am I going to use these secure-variable-secrets?
At the moment one of my pipeline steps gets the latest token value from Azure Key vault and then it's passed into the plan - how can I still get the value from key vault and then supply it as an environment variable for the terraform task?

Thanks for your reply btw!

and apologies @Nullh - I missed your message, but yes, still not working at this point - but partly due to other priorities

[magodo]

@jaredfholgate Thank you for the further clarification for the intent of this issue at #34322, please allow me make a summary:

1. Update the underlying azure sdk in the backend to match the provider. This brings several benefits:

   1. (The original intent of this issue) Eliminate the issue about using SP for Azure CLI auth as this has been enabled in the sdk used by the provider.
   2. Solving any other issues relating to the old sdk, e.g. "Identity not found" error with managed identity in GitHub Action terraform-provider-azurerm#27927

2. For solving token expiry issue. My understanding is that main trick here is to avoid using the OIDC token directly, but using the OIDC request token & URL to get the OIDC token first. This eliminates the token refresh issue as the request token is always guaranteed to be valid (though I failed to find documents for either github action or ADO for this guarantee).

   Especially, for the OIDC request token:

   • Github Action (more info):

     • OIDC request token: Source from env ACTIONS_ID_TOKEN_REQUEST_TOKEN, which is handled by the provider (we shall also handle it in the backend)
     • OIDC request URL: Source from env ACTIONS_ID_TOKEN_REQUEST_URL, which is handled by the provider (we shall also handle it in the backend)
     • OIDC token from response: It is from the value field in the response

   • Azure Pipeline (more info):

     • OIDC request token: Source from env SYSTEM_ACCESSTOKEN (no first class support at provider)
     • OIDC request URL: Constructed as ${SYSTEM_OIDCTOKENURI}?api-version=7.1&serviceConnectionId=${serviceConnectionId} (no first class support at provider)
     • OIDC token from response: It is from the oidcToken field in the response

   With above, for the Azure Pipeline, I think we don't necessarily have to use the Azure official Go SDK (as currently we are not) as it seems to be just a wrapper merging the two step process at the API level, while internally it is still doing the same thing. Instead, users just need to pass in the correct values of OIDC request token and url, it shall just work. Alternatively, users can use the TerraformTaskV4 task as is suggested by the provider: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/guides/service_principal_oidc.

3. The other thing people may come here for is support for using different identities for provider and backend auth. This shall be totally different topic and worth a new issue/feature request.

[jaredfholgate]

Hi @jaredfholgate - yes I am passing it in as a CLI arg. To use an env var am I going to use these secure-variable-secrets? At the moment one of my pipeline steps gets the latest token value from Azure Key vault and then it's passed into the plan - how can I still get the value from key vault and then supply it as an environment variable for the terraform task?

Thanks for your reply btw!

and apologies @Nullh - I missed your message, but yes, still not working at this point - but partly due to other priorities

Hi. I am referring to supplying via the env vars in the provider. I'm not sure the Jason Johnson task currently supports that. I added it to the DevLabs task if you can use that instead? https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks

It is the Use Env Vars for Authentication setting. Set it to true like this: https://github.com/Azure-Samples/azure-devops-terraform-oidc-ci-cd/blob/5e352f715c8ad249ad0a69feff9aa2ff20095e05/pipelines/oidc.yml#L66

If you are just using Azure CLI, then you can set the env vars directly like this: https://github.com/Azure/accelerator-bootstrap-modules/blob/e848f90815c32eabc7c7a22a0fa2ad6da507695b/alz/azuredevops/pipelines/terraform/templates/helpers/terraform-init.yaml#L30

[jaredfholgate]

Hi @jaredfholgate - yes I am passing it in as a CLI arg. To use an env var am I going to use these secure-variable-secrets? At the moment one of my pipeline steps gets the latest token value from Azure Key vault and then it's passed into the plan - how can I still get the value from key vault and then supply it as an environment variable for the terraform task?
Thanks for your reply btw!
and apologies @Nullh - I missed your message, but yes, still not working at this point - but partly due to other priorities

Hi. I am referring to supplying via the env vars in the provider. I'm not sure the Jason Johnson task currently supports that. I added it to the DevLabs task if you can use that instead? https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks

It is the Use Env Vars for Authentication setting. Set it to true like this: https://github.com/Azure-Samples/azure-devops-terraform-oidc-ci-cd/blob/5e352f715c8ad249ad0a69feff9aa2ff20095e05/pipelines/oidc.yml#L66

If you are just using Azure CLI, then you can set the env vars directly like this: https://github.com/Azure/accelerator-bootstrap-modules/blob/e848f90815c32eabc7c7a22a0fa2ad6da507695b/alz/azuredevops/pipelines/terraform/templates/helpers/terraform-init.yaml#L30

To confirm, they are hard-coded as CLI args in the JJ task, so would require an update I think: https://github.com/jason-johnson/azure-pipelines-tasks-terraform/blob/756f2ba62a644865933be62c2f5f94cdd36a9e75/tasks/terraform-cli/src/backends/azurerm.ts#L78

@jason-johnson FYI