JUnit 5 @Order : Test Execution Order Example

In JUnit 5, tests typically should not rely on the order in which they are executed. But there may be situations where test execution ordering becomes an important factor for testing the application functionality such as integration testing.

For example, we may want integration testing for a user management module, where the tests should execute as follows: ‘create a new user -> update the user -> and then delete the user‘.

In JUnit 5, we can order the tests using the following annotations:

@TestClassOrder(ClassName.class) // Orders classes alphabetically
public class OrderedTestClasses {

  @Nested
  @TestMethodOrder(DisplayName.class)  // Orders methods alphanumerically by their names
  class FirstTestClass {

    //...
  }

  @Nested
  @TestMethodOrder(OrderAnnotation.class) // Orders methods using @Order annotation
  class SecondTestClass {

    @Test
    @Order(1)
    void firstTest() {
        //...
    }

    @Test
    @Order(2)
    void secondTest() {
        //...
    }
  }
}

1. Test Methods Ordering using @TestMethodOrder

To control the order in which test methods are executed, use the @TestMethodOrder annotation and specify the desired MethodOrderer implementation.

@TestMethodOrder(DisplayName.class)
public class OrderedTestsDemo {
  //...
}

JUnit 5 (checked with 5.8.1) supports the following inbuilt implementations of the interface MethodOrderer:

• DisplayName – sorts tests alphanumerically based on their display names.
• MethodName – sorts tests alphanumerically based on their names and formal parameter lists.
• OrderAnnotation – sorts tests based on the @Order annotation. Tests with the same order will be sorted arbitrarily adjacent to each other. Not ordered tests will appear at the end.
• Random – orders tests pseudo-randomly. The randomness of tests in each execution can be controlled by property junit.jupiter.execution.order.random.seed.
• Custom – orders the test with the provided ordering logic.

The Alphanumeric class has been marked deprecated and will be removed in JUnit 6. So we can avoid using it.

1.1. MethodOrderer.MethodName

An example of sorting the tests based on their method names.

@TestMethodOrder(MethodOrderer.MethodName.class)
public class MethodNameOrderedTests 
{
    @Test
    void testE() {
        assertTrue(true);
    }

    @Test
    void testA() {
        assertTrue(true);
    }

    @Test
    void testD() {
        assertTrue(true);
    }

    @Test
    void testC() {
        assertTrue(true);
    }

    @Test
    void testB() {
        assertTrue(true);
    }
}

testA()
testB()
testC()
testD()
testE()

1.2. MethodOrderer.DisplayName

An example of sorting the tests based on their display names.

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(MethodOrderer.DisplayName.class)
public class DisplayNameOrderedTests
{
    @DisplayName("5")
    @Test
    void testE() {
        assertTrue(true);
    }

    @DisplayName("3")
    @Test
    void testA() {
        assertTrue(true);
    }

    @DisplayName("1")
    @Test
    void testD() {
        assertTrue(true);
    }

    @DisplayName("2")
    @Test
    void testC() {
        assertTrue(true);
    }

    @DisplayName("4")
    @Test
    void testB() {
        assertTrue(true);
    }
}

1
2
3
4
5

1.3. MethodOrderer.OrderAnnotation

An example of sorting the tests based on their ordering as mentioned in @Order annotation.

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class OrderAnnotationTests {
    @Order(5)
    @Test
    void testE() {
        assertTrue(true);
    }

    @Order(4)
    @Test
    void testA() {
        assertTrue(true);
    }

    @Order(3)
    @Test
    void testD() {
        assertTrue(true);
    }

    @Order(2)
    @Test
    void testC() {
        assertTrue(true);
    }

    @Order(1)
    @Test
    void testB() {
        assertTrue(true);
    }
}

testB()
testC()
testD()
testA()
testE()

1.4. MethodOrderer.Random

An example of sorting the tests randomly. By default, the random seed used for ordering methods is the System.nanoTime() which is generated during the static initialization of the test class.

We can change the randomness by providing a custom seed through property junit.jupiter.execution.order.random.seed.

junit.jupiter.execution.order.random.seed=9999

@TestMethodOrder(MethodOrderer.Random.class)
public class RandomOrderedTests {
    @Test
    void testE() {
        assertTrue(true);
    }

    @Test
    void testA() {
        assertTrue(true);
    }

    @Test
    void testD() {
        assertTrue(true);
    }

    @Test
    void testC() {
        assertTrue(true);
    }

    @Test
    void testB() {
        assertTrue(true);
    }
}

By default, the output will be different for each execution of the test class. But if we use a fixed seed using the property files, the order of the tests will be fixed and will not change between different executions.

The given output is after we have specified the custom seed 9999.

//Execution 1

testB()
testC()
testE()
testA()
testD()


//Execution 2

testB()
testC()
testE()
testA()
testD()

1.5. Custom Ordering

It is possible to define your own custom ordering by implementing the interface MethodOrderer.

In the given example, we are executing the deprecated tests in the end.

import java.util.Comparator;
import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;

public class DeprecatedInEndTestOrder implements MethodOrderer {

    private Comparator<MethodDescriptor> comparator = Comparator
                .comparing(md -> md.getMethod().isAnnotationPresent(Deprecated.class));

    @Override
    public void orderMethods(MethodOrdererContext context) {

        context.getMethodDescriptors().sort(comparator);
    }
}

import static org.junit.Assert.assertTrue;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(DeprecatedInEndTestOrder.class)
public class CustomOrderTests {
    @Test
    @Deprecated
    void testC() {
        assertTrue(true);
    }

    @Test
    void testA() {
        assertTrue(true);
    }

    @Test
    void testD() {
        assertTrue(true);
    }
    @Deprecated
    @Test
    void testE() {
        assertTrue(true);
    }

    @Test
    void testB() {
        assertTrue(true);
    }
}

Notice the test output. The deprecated tests have been executed in the last.

testA()
testB()
testD()
testC()
testE()

2. Test Classes Ordering using @TestClassOrder

Reiterating the fact that test classes typically should not rely on the execution order, there are times when it may be needed.

2.1. Supported Ordering Types

JUnit 5 (checked with 5.8.1) supports the following ways to order tests in the interface ClassOrderer. We can use the class order in the same way as method ordering seen in the above sections.

• ClassName – sorts test classes alphanumerically based on their fully qualified class names.
• DisplayName – sorts test classes alphanumerically based on their display names.
• OrderAnnotation – sorts test classes numerically based on values specified via the @Order annotation.
• Random – orders test classes pseudo-randomly. Supports custom seed using the property junit.jupiter.execution.order.random.seed.

2.2. Test Class Ordering Example

Given is an example for test class ordering in nested test classes.

@TestClassOrder(ClassOrderer.OrderAnnotation.class)
class OrderedTestClassesExample {

    @Nested
    @Order(1)
    class SetupTests {

        @Test
        void test1() {
        }
    }

    @Nested
    @Order(2)
    class AppFlowTests {

        @Test
        void test2() {
        }
    }
}

SetupTests
    - test1()

AppFlowTests
    - test2()

3. Global Configuration for Test Order

We can define the default test method and test class order using the property configuration. We should set the configuration parameter to the corresponding fully qualified class name (e.g., in src/test/resources/junit-platform.properties):

junit.jupiter.testmethod.order.default = org.junit.jupiter.api.MethodOrderer$OrderAnnotation
junit.jupiter.testclass.order.default = org.junit.jupiter.api.ClassOrderer$OrderAnnotation

4. Conclusion

Let me iterate again that forcing an ordering on the test execution order is not recommended and should be avoided. Still, if you come across such a requirement then use above explained techniques to order the test classes and test methods.

I will recommend using @Order annotation with ClassOrderer.OrderAnnotation and MethodOrderer.OrderAnnotation to have full control over the ordering of tests if the test ordering is such important in your case.

Happy Learning !!

Download Source Code