[BUG]: Warning in build after agent upgrade: Failed to get FF TestManagement.Server.EnableAzureTestPlanTaskFlow Value

[ValentijnMakkenze]

What happened?

After upgrading the build agent to the latest version, we are getting build warnings like this

##[warning]Failed to get FF TestManagement.Server.EnableAzureTestPlanTaskFlow Value. Error: System.AggregateException: One or more errors occurred. (TestManagement.Server.EnableAzureTestPlanTaskFlow)
 ---> Microsoft.VisualStudio.Services.Common.VssServiceException: TestManagement.Server.EnableAzureTestPlanTaskFlow
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.Utils.FeatureFlagService.GetFeatureFlagState(String featureFlagName, Guid serviceInstanceId) in D:\a\_work\1\s\src\Agent.Worker\TestResults\Utils\FeatureFlagService.cs:line 39

Versions

Azure Pipelines Agent 3.241.0

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

Azure DevOps Server (Please specify exact version in the textbox below)

Azure DevOps Server Version (if applicable)

Version Azure DevOps Server 2022.2 (AzureDevopsServer_20240702.1)

Operation system

Windows Server 2016 Standard

Version controll system

git

Relevant log output

No response

[vmapetr]

Hi @ValentijnMakkenze thank you for reporting this issue!
According to the error message, looks like the EnableAzureTestPlanTaskFlow is unavailable for some reason, despite that it still exists on the agent and ADO side.
Could you please provide output from the following URL:
https://dev.azure.com/$ORG_NAME/_apis/FeatureFlags/TestManagement.Server.EnableAzureTestPlanTaskFlow, where $ORG_NAME is the name of your organization in ADO.

[ValentijnMakkenze]

We have a self-hosted environment, I changed the url to this
https://tfs.vicrea.nl/Vicrea/_apis/FeatureFlags/TestManagement.Server.EnableAzureTestPlanTaskFlow,

and got this as response

{
    "$id": "1",
    "innerException": null,
    "message": "",
    "typeName": "Microsoft.TeamFoundation.Framework.Server.InvalidFeatureNameException, Microsoft.TeamFoundation.Framework.Server",
    "typeKey": "InvalidFeatureNameException",
    "errorCode": 0,
    "eventId": 3000
}

[lars-lange]

Same here, we have a self-hosted environmen and we updated to Azure DevOps 2022.1 and AgentVersion 3.243.0

..._apis/FeatureFlags/TestManagement.Server.EnableAzureTestPlanTaskFlow
responsed
{
"$id": "1",
"innerException": null,
"message": "TestManagement.Server.EnableAzureTestPlanTaskFlow",
"typeName": "Microsoft.TeamFoundation.Framework.Server.MissingFeatureException, Microsoft.TeamFoundation.Framework.Server",
"typeKey": "MissingFeatureException",
"errorCode": 0,
"eventId": 3000
}

[vmapetr]

Hi @ValentijnMakkenze,
According to the output it's looks like the TestManagement.Server.EnableAzureTestPlanTaskFlow Feature Flag is not registered on the Server side, expected response should look like this:

{
    "name": "TestManagement.Server.EnableAzureTestPlanTaskFlow",
    "description": "Flag for enabling linkage of manual test point information to automated test result in resultspublish command flow.",
    "uri": "_apis/FeatureFlags/TestManagement.Server.EnableAzureTestPlanTaskFlow",
    "effectiveState": "Off",
    "explicitState": "Undefined"
}

Could you please try to install the latest updates for Azure DevOps server and check if the issue still happens?

In case if the issue still occurs could you please open a ticket on the https://developercommunity.visualstudio.com/AzureDevOpsServerTFS, since this repo is related to the agent issues, and it's look like a problem on the Azure DevOps Server side.

[JeremyJeanson]

Hi @vmapetr,

We are using Azure DevOps Server 2022.2 (AzureDevopsServer_20240702.1). The agent version is 3.242.1.
We are in the same situation

##[warning]Failed to get FF TestManagement.Server.EnableAzureTestPlanTaskFlow Value. Error: System.AggregateException: One or more errors occurred. (TestManagement.Server.EnableAzureTestPlanTaskFlow)

{
    "$id": "1",
    "innerException": null,
    "message": "",
    "typeName": "Microsoft.TeamFoundation.Framework.Server.InvalidFeatureNameException, Microsoft.TeamFoundation.Framework.Server",
    "typeKey": "InvalidFeatureNameException",
    "errorCode": 0,
    "eventId": 3000
}

[ValentijnMakkenze]

I reported it here

https://developercommunity.visualstudio.com/t/Warning-in-all-builds-after-upgrade-2022/10727801

[vmapetr]

Hi @ValentijnMakkenze @JeremyJeanson @lars-lange,
I also contacted the team responsible for the related FF and they provided the following mitigation steps.

To avoid the warning it is required to register the TestManagement.Server.EnableAzureTestPlanTaskFlow Feature Flag on the Server.

Please execute the following queue in the config database.

INSERT INTO tbl_RegistryItems (PartitionId, ParentPath, ChildItem, RegValue)

VALUES
(1,'#\FeatureAvailability\Definitions\TestManagement.Server.EnableAzureTestPlanTaskFlow\', 'Description\', 'Flag for controlling enabling of association to testcase workflows with modified Fully Qualified Name'),
(1,'#\FeatureAvailability\Definitions\TestManagement.Server.EnableAzureTestPlanTaskFlow\', 'Name\', 'TestManagement.Server.EnableAzureTestPlanTaskFlow'),
(1,'#\FeatureAvailability\Definitions\TestManagement.Server.EnableAzureTestPlanTaskFlow\', 'Owner\', 'tfs')

And restart IIS Web App for the Azure DevOps collection.

I also clarified where the mentioned FF will be registered on the Server side, and it will happen only with the next major update for on-prem Server, which is planned for mid 2025. So, to resolve this issue until the next release we are preparing additional PR for the agent that will hide this FF under the agent knob, which will be disabled by default, and can be enabled by setting the environment variable. It will be published soon.

[JeremyJeanson]

Thank you.
I will apply this Monday.

[ValentijnMakkenze]

I registered the featureflag and the warning is now gone, thanks

[michha]

@vmapetr Could you notify in this issue, when a agent version is released, that has this knob?
At least v3.243.1 still has this issue
Thank you

[AkisParousis]

Hello I have faced the similar issue on our self-hosted agent

Can someone guide how can I register the TestManagement.Server.EnableAzureTestPlanTaskFlow Feature Flag on the Server, because I have no idea how to perform it

[chucker]

@AkisParousis you need access to the database. On Windows, you can use SQL Server Management Studio to connect to the server. Then, in Object Explorer, expand Databases, go to the database suffixed _Configuration (in my case, it's Tfs_Configuration, but I imagine it's AzureDevOps_Configuration for newer installations). Select that database, then click New Query in the toolbar. In the status bar in the bottom right, the database name should be shown. If it does not end with _Configuration, you probably clicked the toolbar item from the wrong place. Finally, paste the command. Hit F5 once to execute it.

You can then paste select * from tbl_RegistryItems where ParentPath like '%EnableAzureTestPlanTaskFlow%' in the same editor (delete the other query first), and hit F5 again. This should give you the three rows you've just inserted.