Bug: Platform Landing Zone Deployment Fails for management group permissions issues with Azure DevOps

[jaredfholgate]

Is there an existing issue for this?

• I have searched the existing issues

Infrastructure as Code Type? (Required)

terraform

PowerShell Module Version (Optional)

latest

Bootstrap Module Version (Optional)

latest

Starter Module? (Required)

terraform - complete_multi_region

Starter Module Version (Optional)

latest

Input arguments of the ALZ-PowerShell-Module (Optional)

N/A

Debug Output/Panic Output (Optional)

╷
│ Error: Failed to retrieve resource
│
│   with module.management_groups.module.management_groups.azapi_resource.management_groups_level_0["alzroot"],
│   on .terraform/modules/management_groups.management_groups/main.management_groups.tf line 1, in resource "azapi_resource" "management_groups_level_0":
│    1: resource "azapi_resource" "management_groups_level_0" {
│
│ reading Resource: (ResourceId
│ "/providers/Microsoft.Management/managementGroups/alzroot" / Api Version
│ "2023-04-01"): GET
│ https://management.azure.com/providers/Microsoft.Management/managementGroups/alzroot
│ --------------------------------------------------------------------------------
│ RESPONSE 403: 403 Forbidden
│ ERROR CODE: AuthorizationFailed
│ --------------------------------------------------------------------------------
│ {
│   "error": {
│     "code": "AuthorizationFailed",
│     "message": "The client '341a5e18-bf81-48ad-9cd6-998313ebc55d' with object id '341a5e18-bf81-48ad-9cd6-998313ebc55d' does not have authorization to perform action 'Microsoft.Management/managementGroups/read' over scope '/providers/Microsoft.Management/managementGroups/alzroot' or the scope is invalid. If access was recently granted, please refresh your credentials."
│   }
│ }
│ --------------------------------------------------------------------------------
│
╵

Expected Behaviour (Required)

It does not fail.

Actual Behaviour (Required)

It fails.

Steps to Reproduce (Optional)

Run the accelerator with the platform_landing_zone module and Azure DevOps.

Important Factoids (Optional)

This appears to be related to access token refresh in Azure DevOps.

References (Optional)

Azure/terraform-azurerm-avm-ptn-alz#157

[jaredfholgate]

This issue was reported over at Azure/terraform-azurerm-avm-ptn-alz#157.

We have been unable to reproduce in our test environments, but there have been 3 reports so far.

It appears to only be a problem with Azure DevOps and the fixes to the AzAPI don't appear to have solved it.

There is ongoing work to improve OIDC auth in the Terraform providers and backend, however I am opening this issue as it may be possible to migrate AzAPI over the the new auth method now, so will see if that has any impact.

FYI: @matt-FFFFFF @Raphael-kainos @paul-e-martin @MatthewGrimshaw

[jaredfholgate]

In order to implement this in the short term, a change is required to the azapi provider. PR raised here: Azure/terraform-provider-azapi#709

[jaredfholgate]

@MatthewGrimshaw We will shortly release a new version of the bootstrap module that includes a fix for this issue. We set the AZAPI_RETRY_GET_AFTER_PUT_MAX_TIME to 60m. It actually takes between 10 and 15 minutes for the permissions to become consistent in testing.

I am closing this issue for now. Work continues on token refresh in the providers which may reduce down the time it takes, but this solves the problem for now.