02 Oct 2015

Unit Test parallel execution of static classes or ServiceLocator

We all love greenfield projects but sometimes, we have to refactor/support the project that are developed without considering the testability in the first place. But it shouldn’t stop you from adding the unit tests.

This post will show you how to mock the classes and static classes. If you can make the code testable, I would recommend to refactor it instead of using those methods below.

Source Code for Test Project: https://github.com/michaelsync/Michael-Sync-s-blog-sample/tree/master/XunitTestsParallel

Mocking Static Classes

The problem with the static class is that you can’t mock them unless your mocking library supports it. Here is the list of mock libraries (both free and commercial) that supports the static class mocking in .NET world.

Let’s say we are using Microsoft Fakes. You can mock the DateTime.Now as below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
// Shims can be used only in a ShimsContext:
using (ShimsContext.Create())
{
    // Arrange:
    // Shim DateTime.Now to return a fixed date:
    System.Fakes.ShimDateTime.NowGet = () => { return new DateTime(fixedYear, 1, 1); };
    // Instantiate the component under test:
    var componentUnderTest = new MyComponent();
    // Act:
    int year = componentUnderTest.GetTheCurrentYear();
    // Assert: 
    // This will always be true if the component is working:
    Assert.AreEqual(fixedYear, year);
}

It’s pretty easy, huh? What about the service locator?

Mocking Service Locator

Mocking the service locator is very easy as well since it allows us to set the mock locator using ServiceLocator.SetLocatorProvider() method.

The example below shows how to mock the service locator using Moq in xunit test.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
public class TestClass1 {

    public TestClass1() {
        var mock = new Mock<IDataStore>();
        mock.Setup(m => m.GetData())
            .Returns(1); /* Returns 1 from TestClass1 */

        var mockServiceLocator = new Mock<IServiceLocator>();
        mockServiceLocator.Setup(ms => ms.GetInstance<IDataStore>())
            .Returns(mock.Object);

        ServiceLocator.SetLocatorProvider(() => mockServiceLocator.Object);
    }

    [Fact]
    private void TestClass1TestMethod1() {
        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(1, data);
    }

    [Fact]
    private void TestClass1TestMethod2() {
        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(1, data);
    }
}

So far so good. What is the problem?

Problem

Test Parallelization

Some test frameworks let you run the tests in parallel. For example, xunit 2.0 which is released early this year (March 16, 2015) has a new feature that let the tests from different classes run in parallel by default. (You can read about it in “Parallelism” in the release note or here “Running Tests in Parallel”. xUnit 2.0 uses the concepts called “Test Collection” to decide which tests can run against each other in parallel. Each test class is a unique test collection and ParallelizeTestCollectionsproperty is set to true by default.

So, the tests that has the dependency on server locator or static class will be failed randomly with test parallelization but the test will work fine when you run it individually. Because what you setup in one test will be overridden the setup from different test class so the assertion will be failed when the overrides happens.

I have TestClass1 in my previous example above. I will add another class called TestClass2 to show you te random test failture.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
public class TestClass2 {
    public TestClass2() {
        var mock = new Mock<IDataStore>();
        mock.Setup(m => m.GetData())
            .Returns(2); /* Returns 2 here */

        var mockServiceLocator = new Mock<IServiceLocator>();
        mockServiceLocator.Setup(ms => ms.GetInstance<IDataStore>())
            .Returns(mock.Object);

        ServiceLocator.SetLocatorProvider(() => mockServiceLocator.Object);
    }

    [Fact]
    private void TestClass2TestMethod1() {
        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(2, data);
    }

    [Fact]
    private void TestClass2TestMethod2() {
        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(2, data);
    }
}

Try running all tests a few times after adding the TestClass2. You will see that at least one test out of four tests will be failed randomly. It is because the locator is a static class and what we setup (let’s say we return 1.) in TestClass1 will be overridden by the setup from TestClass2 so the assertion will be failed.

Random Test Failed

Solution

How do we fix that?

Initially, I looked at Microsoft Fakes which is shipped with Visual Studio and is also the next generation of Moles. According to the concurrency section of this document, it doesn’t have thread affinity so no go for me.

Shim types apply to all threads in the AppDomain and don’t have thread affinity. This is an important fact if you plan to use a test runner that support concurrency: tests involving shim types cannot run concurrently. This property is not enfored by the Fakes runtime.

Then, I look at Telerik’s JuckMock. The paid version of JuckMock supports what they called “elevated-mocking” that allows you to mock the static class. (Note: The free version “JuckMock Lite” doesn’t support the elevated-mocking.)

So, I refactored my test code as below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
public class TestClass1 {
    private readonly Mock<IServiceLocator> mockServiceLocator;

    public TestClass1() {
        var mock = new Mock<IDataStore>();
        mock.Setup(m => m.GetData())
            .Returns(1);

        mockServiceLocator = new Mock<IServiceLocator>();
        mockServiceLocator.Setup(ms => ms.GetInstance<IDataStore>())
            .Returns(mock.Object);
    }


    [Fact]
    private void TestClass1TestMethod1() {
        Telerik.JustMock.Mock.Arrange<IServiceLocator>(() => ServiceLocator.Current)
            .Returns(mockServiceLocator.Object);

        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(1, data);
    }

    [Fact]
    private void TestClass1TestMethod2() {
        Telerik.JustMock.Mock.Arrange<IServiceLocator>(() => ServiceLocator.Current)
                        .Returns(mockServiceLocator.Object);

        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(1, data);
    }
}

public class TestClass2 {
    private readonly Mock<IServiceLocator> mockServiceLocator;

    public TestClass2() {
        var mock = new Mock<IDataStore>();
        mock.Setup(m => m.GetData())
            .Returns(2);

        mockServiceLocator = new Mock<IServiceLocator>();
        mockServiceLocator.Setup(ms => ms.GetInstance<IDataStore>())
            .Returns(mock.Object);
    }

    [Fact]
    private void TestClass2TestMethod1() {
        Telerik.JustMock.Mock.Arrange<IServiceLocator>(() => ServiceLocator.Current)
            .Returns(mockServiceLocator.Object);

        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(2, data);
    }

    [Fact]
    private void TestClass2TestMethod2() {
        Telerik.JustMock.Mock.Arrange<IServiceLocator>(() => ServiceLocator.Current)
            .Returns(mockServiceLocator.Object);

        var data = ServiceLocator.Current.GetInstance<IDataStore>().GetData();
        Assert.Equal<int>(2, data);
    }
}

Edited: I can’t move Mock.Arrange() to the constructor because I think JustMock has a thread affinity and xunit might use the different thread to call the constructor and test method. That’s why I repeat the code in every test method. End Edited.

Before you run the test, there is a little small thing that you need to do. You need to enable the profiler in Telerik menu.

Enable Profiler

The reason is that JustMock is using the [profiling API][5] to enable the elevated mocking. I heard that TypeMock is using the same thing as well. Please read [this post][6] if you want to know more about it.

Now, you can run all tests without any failure.

All tests

You can download the sample test project here “https://github.com/michaelsync/Michael-Sync-s-blog-sample/tree/master/XunitTestsParallel”.

That is. Yes. I am not encouraging you to use this feature. If you have the capacity to remove the service locator or static classes from your project then you should probably do that. Otherwise, using the elevated mocking is one way of solving problem.

Michael Sync