The Art of Mocking in Software Testing

Gil Zilberfeld, Product Manager Typemock. Dror Helper, Technical Lead, Better Place

https://www.typemock.com/; https://www.gilzilberfeld.com ; https://blog.drorhelper.com

One of the challenges developers face when writing unit tests is how to handle external dependencies. In order to run a test, you may need a connection to a fully populated database, or some remote server; or perhaps there is a need to instantiate a complex class created by someone else.

All these dependencies hinder the ability to write unit tests. When such dependencies need a complex setup for the automated test to run, the end result is fragile tests that break, even if the code under test works perfectly.

This article will cover the subject of mocks (also known as test doubles, stubs and fakes, amongst other names) and creating manual mocks vs. using a full-fledged mocking framework.

But first - What are unit tests?

Unit tests are short, quick, automated tests that make sure a specific part of your program works. They are performed by testing a specific functionality of a method or class which has a clear pass/fail condition. By writing unit tests, developers can make sure their code works, before passing it to QA for further testing.

Let's look at a C# class with a method we'd like to test.

public class UserService
{
	public bool CheckPassword(string userName, string password)
	{
	...
	}
}

We can then write a test that checks for a valid user and password, the method it returns true:

[Test]
public void CheckPassword_ValidUserAndPassword_ReturnTrue()
{
	UserService classUnderTest = new UserService();
	bool result = classUnderTest.CheckPassword("user", "pass");
	Assert.IsTrue(result);
}

By writing several unit tests, we can test for various inputs to CheckPassword, we get expected results, and make sure it performs according to specifications.

Good unit tests run quickly and are isolated from other tests. Both these traits are difficult to achieve out of the box.

Writing unit tests: The obstacles

After a few days of writing tests, every developer hits a wall. The problem is that not all code is created equal and not all methods are simple to test. A common case: What happens when the method-under-test requires access to a database, to check if a user entry exists?

Yes, we can set up a database before running the test however it would cost us precious time in which the developer is waiting for his ~1000 tests to run instead of writing code. Would you set up a database specific for every test and then clean it up after the test runs? We wouldn't.

Sometimes, the code-under-test works perfectly but then you need to set up the environment that the tests depend on. This dependency makes your tests very brittle and they could fail. If tests sometimes fail, depending on the environment, you won't trust them, which defeats the point.

The key to solving these issues lies in the usage of mocking. The mocking mechanism replaces the production code behavior (like the database calls), with code that we can control.

Hand-Rolled Mocking

The first mocking experience usually starts out with hand-rolled mocks. These are classes that are coded by the developer and can replace the production objects for the purpose of testing.

Creating your very first mock is simple - all you need is a class that looks like the class you're replacing, which can be done using inheritance (in Object Oriented languages).

Here is the previous example- now with the CheckPassword method fully implemented:

public class UserService
{
	private IDataAccess_dataAccess;
	public UserService(IDataAccess dataAccess)
	{
	_dataAccess = dataAccess;
	}
	public bool CheckPassword(string userName, string password)
	{
	User user = _dataAccess.GetUser(userName);
	if (user != null)
		{if (user.VerifyPassword(password))
			{
			return true;
			}
		}
		return false;
	}
}

Our production code uses external data (Database) in order to store the application's users. Because deploying and populating a database would increase the test's run time and might fail some other solution is in order.

In order to test the CheckPassword method we need to provide an object that would implement IDataAccess, but that we control. Doing so is quite simple: all we need to do is to create a new class that will implement the desired interface, only whilst the real DataAccess class' implementation goes to the database, ours just returns a value we supply:

public class DummyDataAccess : IDataAccess
{
	private User _returnedUser; 
	public DummyDataAccess(User user)
	{
	_returnedUser = user;
	}
	public User GetUser(string userName)
	{
	return _returnedUser;
	}
}

The class we've created does just one thing - it returns a User when called, instead of calling the database. Using this DummyDataAccess we're now able to alter our test to make it pass:

[Test]

public void CheckPassword_ValidUserAndPassword_ReturnTrue()
{
User userForTest = new User("user", "pass");
IDataAccess fakeDataAccess = new DummyDataAccess(userForTest);
UserService classUnderTest = new UserService(fakeDataAccess);
bool result = classUnderTest.CheckPassword("user", "pass");
Assert.IsTrue(result);
}

First we create a User and a DummyDataAccess object that would return that User. Then we create a real UserService (the class we want to test), and supply it with the DummyDataAccess. We then call the method under test, which eventually uses our fake implementation to return the supplied User data.

At first this might look like cheating - it seems that the test does not use the production code fully, and therefore does not really test the system properly. But remember: we're not interested in the working of the data access in this particular unit test; all we want to test is the business logic of the UserService class.

We should also have additional tests; these ones test that we read and write data correctly into our database. These are called integration tests

Using the DummyDataAccess class achieves three goals:

1. Our test does not need external dependencies (i.e. a database) to run.
2. Our test will execute faster because we do not perform an actual query.

So what is all this mocking about?

While you can spend the whole day categorizing and naming different sorts, we'll just mention the classic mocks and stubs.

• A Stub is an object that is used to replace a real component without calling any of the real component functionality.
• A Mock objects is a stub that is also used as an observer point for the test. Using a Mock object, a test can verify that a specific method was called, and can use this information as a pass/fail criterion.

You'll see that people give their own definitions using the same words. Because of the loaded definitions of the two, from this point on we'll call both "fake objects" (or "fakes" for short). While the difference exists, it becomes less apparent in modern mocking frameworks, and just not worth the fuss.

Why not stop at hand-rolled fakes?

At first, using manually written fake objects seems like a good idea. All software developers know how to write code, and implementing an interface or deriving a new class is a no-brainer.

The problem is that the simple fake class created yesterday, becomes a maintenance nightmare today. Here are a couple of reasons why:

1. Adding new methods to an existing interface

lets go back to the example from the beginning of this article. What happens when we add a new method to the IDataAccess interface? We now need to also implement the new method in the fake object (usually we'll have more than one, so we'll need to implement in the other fakes too). As the interface grows, the fake object is forced to add more and more methods that are not really needed for a particular test just so the code will compile. That's usually necessary work, with almost no value.

2. Adding new functionality to a base class

one way around the method limitation is to create a real class and derive the fake object from it, only faking the methods needed for the tests to pass. Sometimes it can prove risky, though.

The problem is that once derived, our fake objects have fields and methods that perform real actions and could cause problems in our tests. A recent example we encountered shows why: A hand rolled fake was inherited from a production class: it had an internal object opening a TCP connection upon creation. This caused very strange failures in my unit tests, until we were able to track it down. In this case, we wasted time because of the way we created the fake object.

3. Adding new functionality to our fake object

as the number of tests increases, we'll be adding more functionality to our fake object. For some tests method X returns null, while for other tests it returns a specific object. As the needs of the tests grow and become distinct, our fake object adds more and more functionality until it becomes so complicated that it may need unit testing of its own.

All of these problems require us to look for a more robust, industry grade solution - namely a mocking framework.

Mocking Frameworks

A mocking framework (or isolation framework) is a 3^rd party library, which is a time saver. In fact, comparing the saving in code lines between using a mocking framework and writing hand rolled mocks, for the same code, can go up to 90%! Instead of creating our fake objects by hand, we can use the framework to create them, with a few API calls. Each mocking framework has a set of APIs for creating and using fake objects, without the user needing to maintain irrelevant details of the specific test - in other words, if a fake is created for a specific class, when that class adds a new method nothing needs to change in the test. One final remark: a mocking framework is just like any other piece of code and does not "care" which unit testing framework is used to write the test it's in.

Mocking framework types

Different frameworks work in different ways - some create a fake object at run-time, others generate the needed code during compilation, and yet others use method interception to catch calls to real objects and replace these with calls to a fake object. Obviously the framework's technology dictates its functionality.

For example: if a specific framework works by creating new objects at run-time using inheritance, then that framework cannot fake static methods and objects that cannot be derived. It's important to understand the differences between frameworks, prior to committing to one. Once you build a large amount of tests, replacing a mocking framework can be expensive.

What can a mocking framework do for me?

Mocking frameworks perform three main functions:

1. Create fake objects
2. Set behavior on fake objects
3. Verify methods were called

In the next examples, we've used Typemock Isolator, a .NET mocking framework. Wikipedia has an extensive list of mocking frameworks sorted by programming language at http://en.wikipedia.org/wiki/List_of_mock_object_frameworks.

1. Creating fake objects

Once a fake object is created, how does it behave? It might return a fake object, or throw an exception when called, depending on the mocking framework used. Most of the time, to use the new fake object in a test, additional code is required to set its behaviors.

2. Setting behavior on fake objects

After creating a fake object, its behavior needs to be configured. The following example takes the code from the beginning of this article and uses a mocking framework to create a set behavior of a fake DataAccess object:

[Test]
public void CheckPassword_ValidUserAndPassword_ReturnTrue()
{
User userForTest = new User("user", "pass");
IDataAccess fakeDA = Isolate.Fake.Instance<IDataAccess>();
Isolate.WhenCalled(() => fakeDA.GetUser(string.Empty)).WillReturn(userForTest)
UserService classUnderTest = new UserService(fakeDataAccess);
bool result = classUnderTest.CheckPassword("user", "pass");
Assert.IsTrue(result);
}

The test is very similar to the test we had before. This time, we don't need to create and maintain a class to fake the DataAccess class. We can create tests without worrying about any maintenance penalty.

3. Verify methods were called

Test frameworks can test state value: fields, properties, variables. Comparing the actual value to the expected ones are the pass/fail criteria. Mocking frameworks add another way to test - checking whether methods were called, in which order, how many times and with which arguments. For example, let's say that a new test is required for adding a new user: if that user does not exist than call IDataAccess.AddNewUser method. Note that the AddUser method doesn't have a return value we can check on. We need to know if the call actually happened.

[Test]
public void AddUser_UserDoesNotExist_AddNewUser()
{
IDataAccess fakeDA = Isolate.Fake.Instance<IDataAccess>();
Isolate.WhenCalled(() => fakeDA.GetUser(string.Empty)).WillReturn(null);
UserService classUnderTest = new UserService(fakeDA);
classUnderTest.AddUser("user", "pass");
Isolate.Verify.WasCalledWithAnyArguments(() => fakeDA.AddUser(null));
}

The test is very similar to the previous test, except for two points:

1. null is returned when GetUser is invoked to specify that the user does not exist
2. At the end of the test, instead of an assertion on a result, the mocking framework is used to verify that a specific method was called.

Some mocking frameworks have additional capabilities, other than the basic main three such as the ability to invoke events on the fake object or cause the creation of a specific fake object inside the product code. The three basic capabilities are the core functionality expected from every mocking framework. Additional features should be compared and checked when deciding which mocking framework to use.

Choosing a mocking framework

Changing an existing mocking framework requires updating all existing unit tests and so choosing the right mocking framework is important. There are no clear rules to how one should decide which framework to use but there are some key factors that need to be taken into consideration:

• Functionality

Not all mocking framework are created equal. Some frameworks might offer additional capabilities that other do - for example if a mocking framework uses inheritance to create fake objects it cannot fake static and non-virtual methods, while a framework that employs instrumentation and/or method interception can. Look for the features that are not strictly "mocking" features such as event and method invocation - those can help write better unit tests faster. Almost every mocking framework has a site or white paper detailing its features. Compare several frameworks features side by side and see which scores higher.

• API and ease of use

Just like any other 3^rd party library it is important that a mocking framework should be easy to use. Try several frameworks to see which makes most sense to you. A simple, discoverable and readable API (application programming interface) is important because it would directly affect how readable the tests are that use it. Easy to use as well as easy to read are definitely factors worthy of consideration.

• Price

Some mocking frameworks are free while others cost money. Usually (but not always) there is a good reason that a certain framework is not given for free. Users might be entitled to premium support or training that would help getting up to speed with the new tool. The paid version might have features that the free frameworks do not have. Check the licensing scheme, keep in mind that you might need to purchase a new license for each developer in the team as well as build servers. At the end of the day it's all about ROI (return on investment) even a pricey tool that saves 50% of the time of each developer is worth the investment.

Best Practices and common pitfalls

When using a mocking solution it's easy to forget that it's merely a tool - and as such can be abused. Just like any other development tool it is up to the developer to learn to use it well.

Here are a couple of aspects you should know how to handle.

1. Know what to isolate

The key advice when using a fake object is to understand what is under test and what is the dependency. The object under test would not usually be faked - so finding the target and scope of the test helps finding out what to fake.

2. Fake as little as needed

Using too many fake objects creates fragile tests - tests that are likely to break when production code changes occur. It is advisable to fake as little as possible. Faking chatty interfaces should be avoided because a small change in the order of calls would break your test. Start by writing the test without the fake objects then fake only what you need to makethe test pass.

3. Fake the immediate neighbors

Fake the objects directly called by the subject of the test. Unless you want to test the interaction between several classes try to limit the scope of the fake object to those directly affecting the class under test.

4. Don't misuse Verify

When you have a fake object, the entire world looks like it should be verified. It's easy to fall to the trap of making sure that method "A" calls method "B" - most of the time it does not really matter. Methods are refactored and changed frequently, so Verify should only be used where it's the only pass/fail criterion of your test.

5. One (or two) assertions per test

When a test has more than one assert it may be testing too many things at the same time. The same principle applies to using Verify. Testing if three different methods where called should be done in three separate tests. If the first verify that fails throws an exception, we don't have any clear knowledge about the success or failure of the other two. This keeps us further from fixing the problem.

Summary

Unit testing is a major component of every agile methodology. The early feedback you receive from your tests help you feel confident that you didn't introduce new bugs, and that you gave QA actual working code.

This article was written to give you an understanding into the world and art of Mocking.

Mocking frameworks are essential tools for writing unit tests. In fact, without a tool like this, you're bound to fail in your effort - either you won't have unit tests that give early feedback, or no tests at all.

This is why deciding on a mocking framework or some other similar solution is as important as deciding the unit testing framework used. Once you pick a framework, master it. It helps make your unit testing experience easy and successful.

---

More Software Testing Knowledge

Software Testing Magazine

Software Testing Videos and Tutorials

---

Click here to view the complete list of archived articles

This article was originally published in the Summer 2011 issue of Methods & Tools