JUnit

Neodymium Test Execution

With Neodymium a test case can run with any set of test data and thus enables Data Driven Testing. Next to the Multi Datasets Neodymium offers an additional support to run the test cases with their Datasets in different browsers. This is the Neodymium Multi Browser support. Therefore a test case may be annotated with any Datasets and Browsers. Before starting the test execution Neodymium determines all tests to be executed using the annotated Datasets and Browsers. Here we provide some details on how this works within Neodymium.

Neodymium implements the extended JUnit4 test runner NeodymiumRunner to provide features like test data handling, multi browser support and the consequential test multiplication.

Neodymium provides an extended JUnit4 test runner. The NeodymiumRunner is used to provide our features like test data handling, multi browser support and the consequential test multiplication.

Usage

The NeodymiumRunner can be used easily by annotating the test case class. Please see example below:

@RunWith(NeodymiumRunner.class)
public class DemoTest
{
    @Test
    public void ensureFunctionality()
    {
        // add test code
    }
}

Supported Annotations

Neodymium provides a set of annotations that can be used to configure a test case using the NeodymiumRunner.

Multi Browser

• @Browser
• @SuppressBrowser

Please check the Multi browser support page for detailed information.

Test Data

• @DataSet
• @SuppressDataSet
• @DataFile

Please check the Test data provider page for detailed information.

Test Execution Order

In general the standard test execution order of JUnit > 4.11 applies. More info here https://github.com/junit-team/junit4/wiki/Test-execution-order. This means by default there is no fixed order within the methods annotated with @Test since they are retrieved as map.

For Neodymium this results in a mixture of ordered and unordered. The test methods are retrieved as map but while computing the test multiplication for test data sets and browsers they are added as complete sets (cross product of method, browsers and test data sets) for each method that is effected by Neodymium's annotations (see above). JUnit's method ordering is applied first. The order of the used browsers is the same as stated within the test. The data sets are executed in the order they are listed within the data file or as specified by using the @DataSet annotation. The browsers are applied before the data sets.

In addition it's possible to force fix the method order by using JUnit's @FixMethodOrder annotation. Please see example below and let's assume we use a data file with 3 data sets:

@RunWith(NeodymiumRunner.class)
@Browser("Chrome_1024x768")
@Browser("Firefox_1024x768")
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class OrderDemoTest
{
    @Test
    public void testC()
    {
        System.out.println("testC with browser: '" + Neodymium.getBrowserProfileName() + "' and data set: '" + Neodymium.dataValue("testId") + "'.");
    }

    @Test
    // this overwrites the order of the data file
    @DataSet(id = "dataSet3")
    @DataSet(id = "dataSet2")
    @DataSet(id = "dataSet1")
    public void testB()
    {
        System.out.println("testB with browser: '" + Neodymium.getBrowserProfileName() + "' and data set: '" + Neodymium.dataValue("testId") + "'.");
    }

    @Test
    // this overwrites the order of the test class
    @Browser("Firefox_1024x768")
    @Browser("Chrome_1024x768")
    public void testA()
    {
        System.out.println("testA with browser: '" + Neodymium.getBrowserProfileName() + "' and data set: '" + Neodymium.dataValue("testId") + "'.");
    }
}

The console output of the code above would look like this:

testA with browser: 'Firefox_1024x768' and data set: 'dataSet1'.
testA with browser: 'Firefox_1024x768' and data set: 'dataSet2'.
testA with browser: 'Firefox_1024x768' and data set: 'dataSet3'.
testA with browser: 'Chrome_1024x768' and data set: 'dataSet1'.
testA with browser: 'Chrome_1024x768' and data set: 'dataSet2'.
testA with browser: 'Chrome_1024x768' and data set: 'dataSet3'.
testB with browser: 'Chrome_1024x768' and data set: 'dataSet3'.
testB with browser: 'Chrome_1024x768' and data set: 'dataSet2'.
testB with browser: 'Chrome_1024x768' and data set: 'dataSet1'.
testB with browser: 'Firefox_1024x768' and data set: 'dataSet3'.
testB with browser: 'Firefox_1024x768' and data set: 'dataSet2'.
testB with browser: 'Firefox_1024x768' and data set: 'dataSet1'.
testC with browser: 'Chrome_1024x768' and data set: 'dataSet1'.
testC with browser: 'Chrome_1024x768' and data set: 'dataSet2'.
testC with browser: 'Chrome_1024x768' and data set: 'dataSet3'.
testC with browser: 'Firefox_1024x768' and data set: 'dataSet1'.
testC with browser: 'Firefox_1024x768' and data set: 'dataSet2'.
testC with browser: 'Firefox_1024x768' and data set: 'dataSet3'.

Please clarify how Surefire treats the @Test classes and if there is any execution order. Often the tests are structured by classes as well as by methods.