Unit testing patterns for concurrent code

Unit testing patterns for concurrent code
Dror Helper
drorh@oz-code.com | @dhelper | http://blog.drorhelper.com
Examples: https://github.com/dhelper/ConcurrentUnitTesting

About.ME
• Senior consultant @CodeValue
• Developing software (professionally) since 2002
• Mocking code since 2008
• Test Driven Developer
• Blogger: http://blog.drorhelper.com

We live in a concurrent world!
The free lunch is over!
• Multi-core CPUs are the new standard
• New(er) language constructs
• New(ish) languages

Meanwhile in the unit testing “world”
[Test]
public void AddTest()
{
var cut = new Calculator();
var result = cut.Add(2, 3);
Assert.AreEqual(5, result);
}

The dark art of concurrent code
Several actions at the same time
Hard to follow code path
Non deterministic execution

Good unit tests must be:
• Trustworthy
– Consistent results
– Only fail due to bug or requirement change
• Maintainable
– Robust
– Easy to refactor
– Simple to update
• Readable

Concurrency test “smells”
× Inconsistent results
× Untraceable fail
× Long running tests
× Test freeze

How can we test this method
public void Start()
{
_worker = new Thread(() => {
while (_isAlive) {
Thread.Sleep(1000);
var msg = _messageProvider.GetNextMessage();
//Do stuff
LastMessage = msg;
}
});
_worker.Start();
}

Testing start take #1
[TestMethod]
public void ArrivingMessagePublishedTest()
{
var fakeMessageProvider = A.Fake<IMessageProvider>();
A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!");
var server = new Server(fakeMessageProvider);
server.Start();
Thread.Sleep(2000);
Assert.AreEqual("Hello!", server.LastMessage);
}

Test smell - “Sleep” in test
× Time based - fail/pass inconsistently
× Test runs for too long
× Hard to investigate failures

“In concurrent programming if
something can happen, then sooner
or later it will, probably at the most
inconvenient moment”
Paul Butcher – Seven concurrency models in seven weeks

Testing start take #2
[TestMethod]
public async Task ArrivingMessagePublishedTest()
{
var server = new Server(fakeMessageProvider);
server.Start();
await Task.Delay(2000);
Assert.AreEqual("Hello!", server.LastMessage);
}

Solution: avoid concurrent code!

Pattern #1 – humble object pattern
We extract all the logic from the hard-to-test component into a component
that is testable via synchronous tests.
http://xunitpatterns.com/Humble%20Object.html
Perform Action
Code under Test
Start
Humble object
Async
Perform action
Assert Result
Production
Code

public void Start()
{
while (_isAlive) {
Thread.Sleep(1000);
var msg = _messageProvider.GetNextMessage();
//Do stuff
LastMessage = msg;
}
});
_worker.Start();
}

public void Start()
{
while (_isAlive) {
Thread.Sleep(1000);
_messageHandler.HandleNextMessage();
}
});
_worker.Start();
}

And finally – the test
[TestMethod]
public void ArrivingMessagePublishedTest()
{
var messageHandler = new MessageHandler(fakeMessageProvider);
messageHandler.HandleNextMessage();
Assert.AreEqual("Hello!", messageHandler.LastMessage);
}

Concurrency as part of program flow
public class MessageManager
{
private IMesseageQueue _messeageQueue;
public void CreateMessage(string message)
{
// Here Be Code!
_messeageQueue.Enqueue(message);
}
}
public class MessageClient
{
private IMesseageQueue _messeageQueue;
public string LastMessage { get; set; }
private void OnMessage(object sender, EventArgs e)
{
// Here Be Code!
LastMessage = e.Message;
}
}

Test before – Test After
Start
Code under test
Async
Logic2
Assert results
Logic1
Start Logic1 Fake Assert Results
Start Fake Logic2 Assert Results

Testing Flow part 1
[TestMethod]
public void AddNewMessageProcessedMessageInQueue()
{
var messeageQueue = new AsyncMesseageQueue();
var manager = new MessageManager(messeageQueue);
manager.CreateNewMessage("a new message");
Assert.AreEqual(1, messeageQueue.Count);
}

Testing Flow part 2
[TestMethod]
public void QueueRaisedNewMessageEventClientProcessEvent()
{
var messeageQueue = new AsyncMesseageQueue();
var client = new MessageClient(messeageQueue);
client.OnMessage(null, new MessageEventArgs("A new message"));
Assert.AreEqual("A new message", client.LastMessage);
}

Avoid concurrency Patterns
The best possible solution
No concurrency == no problems
× Do not test some of the code
× Not applicable in every scenario

How can we test this class?
public class ClassWithTimer
{
private Timer _timer;
public ClassWithTimer(Timer timer)
{
_timer = timer;
_timer.Elapsed += OnTimerElapsed;
_timer.Start();
}
private void OnTimerElapsed(object sender, ElapsedEventArgs e)
{
SomethingImportantHappened = true;
}
public bool SomethingImportantHappened { get; private set; }
}

Not a good idea
[TestMethod]
public void ThisIsABadTest()
{
var timer = new Timer(1);
var cut = new ClassWithTimer(timer);
Thread.Sleep(100);
Assert.IsTrue(cut.SomethingImportantHappened);
}

Set timeout/interval to 1
Also seen with a very small number (or zero)
Usually done when need to wait for next tick/timeout
× Time based == fragile/inconsistent test
× Hard to investigate failures
× Usually comes with Thread.Sleep

Fake & Sync
Test
Code under Test
Fake
Logic
Assert Results

Using Typemock Isolator to fake Timer
[TestMethod, Isolated]
public void ThisIsAGoodTest()
{
var fakeTimer = Isolate.Fake.Instance<Timer>();
var cut = new ClassWithTimer(fakeTimer);
var fakeEventArgs = Isolate.Fake.Instance<ElapsedEventArgs>();
Isolate.Invoke.Event(
() => fakeTimer.Elapsed += null, this, fakeEventArgs);
}

Unfortunately not everything can be faked
• Mocking tool limitation (example: inheritance based)
• Programming language attributes
• Special cases (example: MSCorlib)
Solution – wrap the unfakeable
× Problem – it requires code change

Wrapping Threadpool
public class ThreadPoolWrapper
{
public static void QueueUserWorkItem(WaitCallback callback)
{
ThreadPool.QueueUserWorkItem(callback);
}
}

Wrapping Threadpool
public class ClassWithWrappedThreadpool
{
public void RunInThread()
{
ThreadPool.QueueUserWorkItem(_ =>
{
SomethingImportantHappened = true;
});
}
public bool SomethingImportantHappened { get; private set; }
}
ThreadPoolWrapper.QueueUserWorkItem(_ =>

Testing with new ThreadpoolWrapper
[TestClass]
public class WorkingWithThreadpool
{
[TestMethod, Isolated]
public void UsingWrapperTest()
{
Isolate.WhenCalled(() => ThreadPoolWrapper.QueueUserWorkItem(null))
.DoInstead(ctx => ((WaitCallback)ctx.Parameters[0]).Invoke(null));
var cut = new ClassWithWrappedThreadpool();
cut.RunInThread();
}
}

How can we test that an
asynchronous operation
never happens?

Test Code Under Test
Is Test
Async
Deterministic Assert Results
Run in sync

Another day – another class to test
public void Start()
{
_cancellationTokenSource = new CancellationTokenSource();
Task.Run(() =>
{
var message = _messageBus.GetNextMessage();
if(message == null)
return;
// Do work
if (OnNewMessage != null)
{
OnNewMessage(this, EventArgs.Empty);
}
}, _cancellationTokenSource.Token);
}

Switching code to “test mode”
• Dependency injection
• Preprocessor directives
• Pass delegate
• Other

Run in single thread patterns
• Fake & Sync
• Async code  sync test

Synchronized run patterns
When you have to run a concurrent test in a predictable way

The Signaled pattern
Start
Run
Code under test
Signal
Fake object
Call
Wait Assert result

Using the Signaled pattern
public void DiffcultCalcAsync(int a, int b)
{
Task.Run(() =>
{
Result = a + b;
_otherClass.DoSomething(Result);
});
}

Using the Signaled pattern
[TestMethod]
public void TestUsingSignal()
{
var waitHandle = new ManualResetEventSlim(false);
var fakeOtherClass = A.Fake<IOtherClass>();
A.CallTo(() => fakeOtherClass.DoSomething(A<int>._)).Invokes(waitHandle.Set);
var cut = new ClassWithAsyncOperation(fakeOtherClass);
cut.DiffcultCalcAsync(2, 3);
var wasCalled = waitHandle.Wait(10000);
Assert.IsTrue(wasCalled, "OtherClass.DoSomething was never called");
Assert.AreEqual(5, cut.Result);
}

Busy Assert
Start
Run
Code under
test
Assert or
Timeout

Busy assertion
[TestMethod]
public void DifficultCalculationTest()
{
var cut = new ClassWithAsyncOperation();
cut.RunAsync(2, 3);
AssertHelper.BusyAssert(() => cut.Result == 5, 50, 100, "fail!");
}

Synchronized test patterns
× Harder to investigate failures
× Cannot test that a call was not made
Test runs for too long but only when it fails
 Use if other patterns are not applicable

Unit testing for concurrency issues
Because concurrency (and async) needs to be tested as well

Test for async
Start Method under test
Fake object
wait
Assert results

Test for Deadlock
Start
Execute
Code under
test
Thread
WWaaitit Assert result
Call Thread

[TestMethod, Timeout(5000)]
public void CheckForDeadlock()
{
var fakeDependency1 = A.Fake<IDependency>();
var fakeDependency2 = A.Fake<IDependency>();
var waitHandle = new ManualResetEvent(false);
var cut = new ClassWithDeadlock(fakeDependency1, fakeDependency2);
A.CallTo(() => fakeDependency1.Call()).Invokes(() => waitHandle.WaitOne());
A.CallTo(() => fakeDependency2.Call()).Invokes(() => waitHandle.WaitOne());
var t1 = RunInOtherThread(() => cut.Call1Then2());
var t2 = RunInOtherThread(() => cut.Call2Then1());
waitHandle.Set();
t1.Join();
t2.Join();
}
T1
T2
L1
L2

Avoid
Concurrency
Humble
object
Test before –
test after
Run in single
thread
Fake & Sync
Async in
production -
sync in test
Synchronize
test
The Signaled
pattern
Busy
assertion
Concurrency
tests
Test for
Deadlock
Test for non
blocking
Concurrent unit testing patterns

Conclusion
It is possible to test concurrent code
Avoid concurrency
Run in single thread
Synchronize test

Dror Helper
C: 972.50.7668543
e: drorh@codevalue.net
B: blog.drorhelper.com
w: www.codevalue.net

Unit testing patterns for concurrent code

More Related Content

Similar to Unit testing patterns for concurrent code (20)

More from Dror Helper (20)

Recently uploaded (20)

Unit testing patterns for concurrent code

Editor's Notes