Cannot read properties of undefined (reading '0')

[quotschmacher]

Description

The XML xUnit file cannot be processed.

steps:
- task: bcook.azdevops-testplan-extension.publishtestplanresults.PublishTestPlanResults@0
  displayName: 'Publish Test Results'
  inputs:
    accessToken: '$(System.AccessToken)'
    testPlan: 'UI Test mit Testcafe'
    testResultFormat: xUnit
    testResultFiles: testcafe/report.xml
    testCaseMatchStrategy: name

Environment Details

• Pool: Azure Pipelines
• Image: ubuntu-22.04
• Queued: Today at 11:24
• Agent: Hosted Agent
• • pipeline-format: Designer-based (Classic)

The xUnit file

• report file generated with the Testcafe xUnit reporter

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite name="TestCafe Tests: Firefox 122.0 / Ubuntu 22.04" tests="2" failures="0" skipped="0" errors="0" time="211.682" timestamp="Tue, 13 Feb 2024 10:28:58 GMT" >
  <testcase classname="Neuland Tests" file="/home/vsts/work/1/s/testcafe/tests/testNld.js" name="LogIn" time="15.513">
  </testcase>
  <testcase classname="Neuland Tests" file="/home/vsts/work/1/s/testcafe/tests/testNld.js" name="Auftrag erfassen" time="193.695">
  </testcase>
</testsuite>

Pipeline Output

provide output from the build with diagnostics enabled (system.debug variable set to 'true'). Mask secrets or sensitive information with xxx:

##[debug]Evaluating condition for step: 'Publish Test Results'
##[debug]Evaluating: succeeded()
##[debug]Evaluating succeeded:
##[debug]=> True
##[debug]Result: True
Starting: Publish Test Results
==============================================================================
Task         : Publish Test Plan Results
Description  : Publishes cross-platform test results (xUnit, JUnit) to an Azure DevOps Test Plan
Version      : 0.1.10
Author       : Bryan Cook
Help         : 
==============================================================================
##[debug]Using node path: /home/vsts/agents/3.232.3/externals/node16/bin/node
[...]
Available Test Points: 2
##[debug]reading TestFrameworkParameters from task inputs.
##[debug]testResultFormat=xUnit
##[debug]testResultDirectory=undefined
##[debug]testResultFiles=/home/vsts/work/1/s/testcafe/report.xml
##[debug]check path : /home/vsts/work/1/s/testcafe/report.xml
##[debug]converting test framework results into unified format
##[debug]task result: Failed
##[error]Cannot read properties of undefined (reading '0')
##[debug]Processed: ##vso[task.issue type=error;]Cannot read properties of undefined (reading '0')
##[debug]Processed: ##vso[task.complete result=Failed;]Cannot read properties of undefined (reading '0')

[quotschmacher]

Just looked up some guideline about xUnit and found out that my xml file looks more like jUnit. But setting the task up to interprete a jUnit file gives the same error.

edit:

The more I look at this here https://github.com/testmoapp/junitxml?tab=readme-ov-file#complete-junit-xml-example the more I think the problem is a not correct implementation of the xml file generation...
But: DevOps can handle this xml and show the result of the test run.

[bryanbcook]

The 3rd party library I'm using to read through the test results doesn't like the file that you're providing.

The xml you supplied is definitely not xUnit. Take a look at some of the sample test data in the source code: PublishTestPlanResultsV1/test/data.

I took your sample xml and wrote a quick unit test to see what is breaking when the file is parsed. It looks like the JUnit parser is expecting a top-level node <testSuites>. Your file works fine if I add this root node.

I think the challenge is there is no formal JUnit schema defined in the JUnit project. I've identified two reference examples:

• https://github.com/testmoapp/junitxml - defines <testsuites> as a root node. This is the format used by the testparser project
• https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd - referenced by IBM's website defines an xml schema that supports either <testsuites> or <testsuite> as the root node.

My research suggests that the java implementation has a legacy reporter that generates an xml file per test suite, and an extra step is required to aggregate the files into a single xml file. The java implementation also has an Open Test Reporter format that generates a single file with <testsuites>.

I can open a defect in the testparser project to add support for mixed formats (<testsuites> | <testsuite>), but we won't have a resolution for that until that dependency is resolved.

In the meantime, you might need to add a small script that can massage your xml. This would be temporary until the testparser is resolved. Something like this powershell task:

# create a top-level node
$root = [xml]"<testsuites count='0' />"
# read the existing xml file into memory
$existing = [xml](Get-Content -Path testcafe/result.xml)
# import the node into the same document namespace
$testSuite = $root.ImportNode( $existing.testsuite, $true )
# add the imported element
$root.testSuites.AppendChild( $testSuite )
# save the changes
$root.Save( 'testcafe/result.xml' )

[quotschmacher]

thanks for your research and effort!

your first reference example states the following at https://github.com/testmoapp/junitxml?tab=readme-ov-file#complete-junit-xml-example :

Usually the root element of a JUnit XML file. Some tools leave out
the element if there is only a single top-level element (which
is then used as the root element).

azure dev ops is for example such a tool that excepts it without this testsuites root-element.

i will see what i can do but as this is not your fault i will close this ticket. thanks a lot - i will go on trying to get your extension running for me ;-)

[bryanbcook]

Does TestCafe emit the testsuites root element if there's more than one suite?

[quotschmacher]

it emits it in any case...

https://github.com/alexschwantes/testcafe-reporter-junit/blob/master/src/index.js -> there is no "testsuites"-tag.

i will look if i can write my own reporter - does not look that complicated...

[bryanbcook]

I wouldn't worry about creating a custom reporter. As per the thread above, I've submitted a PR to the test-parser library that this project uses. The PR has been merged, but I need the owner to publish the changes to the npm repository. The work item for that is here: test-results-reporter/parser#46

As soon as the owner of that project creates a release with the changes, I can update publish a new version of the extension with the updated dependency..

[bryanbcook]

@quotschmacher - the test-parser dependency has been updated to 0.1.7 which includes support for <testsuite>.

Please advise if you have any further issues. Closing this issue.

[bryanbcook]

@quotschmacher - following up. Were you able to publish your results ok?