Click here to Skip to main content
15,868,016 members
Articles / Programming Languages / C#

The Dynamic Duo of Unit Testing – xUnit.net and AutoFixture

Rate me:
Please Sign up or sign in to vote.
4.92/5 (20 votes)
8 Jul 2015CPOL9 min read 55.9K   398   23   15
This article introduces two free, open source, tools that will make your unit tests more declarative and compact and a lot easier to read and understand.

Introduction

Quote:

Worried that TDD will slow down your programmers? Don't. They probably need slowing down.

J. B. Rainsberger

Admittedly, for many years, in my own world, test-driven development (TDD) and unit-testing was something “the others” did.

Then, a few years ago, I started to pay interest to Dependency Injection (DI) as a method for ensuring loose coupling and high maintainability of my code. It turns out that one of the by-products of using DI is that it makes your code base much more testable. Since then, it actually became somewhat natural for me to use unit testing more actively.

The "real" dynamic duo (Batman and Robin)

That’s when I was first introduced to “the dynamic duo” of unit testing – the unit testing framework xUnit.net and the automation tool AutoFixture. Both of them are free, open source, community-focused tools that are available through the NuGet package manager Visual Studio extension.

I won’t claim to do strict TDD (red-green-refactor), but these days unit testing has become an integrated part of my process. At the end of the day, I strive to have full code coverage of members in domain and data layers.

In this article, I will provide a short introduction to the power of the dynamic duo through a few usage samples.

The code base that I am unit testing in this article is a slightly simplified version of a lightweight domain services library – previously introduced in another CodeProject article of mine.

xUnit.net

You might wonder why I find it beneficial to use xUnit.net instead of Microsoft’s own MSTest, which comes as an integrated part of Visual Studio. There are many good reasons for this – for example described in this article. Luckily, test lists (.vsmdi files) are deprecated in MSTest since Visual Studio 2012. However, the sole fact that xUnit.net (unlike MSTest) is extensible - which allows third party tools such as AutoFixture to provide useful extensions to xUnit.net - makes the shift worthwhile.

For an overview of xUnit.net functionality and differences compared to MSTest, I like this comparison sheet.

Data-driven Tests with xUnit.net

xUnit.net has very nice support for parameterized tests (also called data-driven tests). To use data-driven tests in xUnit.net, simply create a new project in Visual Studio and add a reference to xUnit.net using the NuGet package manager.

xUnit.net in NuGet Package Manager

Notice that, unlike for MSTest, it is not necessary to create a specific unit test project.

To be able to run the xUnit.net tests using the Visual Studio Test Explorer, install the xUnit.net Runner NuGet package.

xUnit.net runner in NuGet package manager

While regular xUnit.net test methods are identified by applying the [Fact] attribute, data-driven tests use the [Theory] attribute. Theories are used for tests that are only true for a given set of input data. So, part of testing a theory is providing sets of compliant data. This way, a single test definition can be invoked multiple times at run time.

Let’s create a number of test cases for the GetRoles method in the AccountService class. Each account has a set of roles. There are 4 different roles defined like this:

C#
[Flags]
public enum Roles
{
    Guest = 1,
    User = 2,
    Editor = 4,
    Administrator = 8
}

The GetRoles method is supposed to return the role set represented as an array of strings.

Using the [InlineData] attribute, you can specify test data that is passed to the parameters of the test method during runtime:

C#
[Theory]
[InlineData(Roles.Guest | Roles.User | Roles.Editor, new[] { "Guest", "User", "Editor" })]
[InlineData(Roles.Administrator, new[] { "Administrator" })]
[InlineData(Roles.Guest | Roles.Editor, new[] { "Guest", "Editor" })]
[InlineData(Roles.Editor | Roles.Guest, new[] { "Guest", "Editor" })]
public void GetRolesIsOk(Roles roles, string[] expected)
{
    var account = new Account(Guid.NewGuid(), "John Doe") {Roles = roles};
    var accountService = new AccountService(new FakeAccountRepository());
    accountService.Add(account);

    Assert.Equal(expected, accountService.GetRoles(account.Id));
}

This test will execute 4 times, once for each of the [InlineData] attributes. For an account with a role set of for example Roles.Guest|Roles.Editor, the GetRoles method must return an array with elements “Guest” and “Editor”.

xUnit.net offers several other options for defining data driven tests. The [InlineData] attribute is just one of a handful of extensions of the abstract DataAttribute class, which represents a data source for a data theory. Other DataAttribute extensions are the [ClassData], [PropertyData], [ExcelData] and [OleDbData] attributes.

The abstract DataAttribute class is one of the well-defined extensibility points of xUnit.net that is utilized by AutoFixture, but more about that in a moment.

AutoFixture

Generally, when writing unit tests, one of the more tedious and time consuming tasks is to set up the test fixture. The test fixture is all the things that must be in place in order to run a test and expect a particular outcome. In the above test, the test fixture setup involves creating an account, setting the necessary properties on the account (the roles), establishing the account service and adding the account.

However, very often you do not need any specific data in your test. Maybe you just need a random list of strings or a repository of entities with random properties. This is where AutoFixture comes in. AutoFixture can provide such anonymous variables for you. It can create values of virtually any type without the need for you to explicitly define which values should be used.

Even more interesting, AutoFixture hooks into the previously mentioned DataAttribute extensibility point of xUnit.net by providing the awesome [AutoData] attribute.

As the name implies, this attribute will provide auto-generated data for xUnit.net data theories.

On a side note, AutoFixture is by no means tightly coupled to xUnit.net. AutoFixture can be used with any unit testing framework. However, AutoFixture provides an extension that nicely leverages the data theory feature of xUnit.net.

To enable this functionality, you must add AutoFixture with xUnit.net v2 data theories to your test project using the NuGet package manager.

Autofixture with xunit extensions in NuGet package manager

Now, let’s look at an example of an automatic fixture setup for testing the AccountService class, which is a service for managing user accounts.

The AccountService class uses dependency injection and requires an account repository to be injected through the constructor:

C#
public class AccountService : BaseService<Account>
{
    public AccountService(IRepository<Account> repository)
        : base(repository)
    {
    }

    ...
}

For this purpose, you can create an instance of the FakeAccountRepository. The FakeAccountRepository inherits the generic FakeRepository<TEntity> where "persistence” is done in an in-memory object (a Dictionary object). This is obviously useless in a real application but perfect as a fake repository for unit testing.

C#
internal class FakeAccountRepository : FakeRepository<Account>, IRepository<Account>
{
    ...
}

Furthermore, you might also like the fixture to include population of the fake repository with a few random accounts.

The above fixture setup can be automated in a custom extension of the AutoFixture [AutoData] attribute particularly targeted against testing of the account service functionality.

C#
internal class AutoAccountDataAttribute : AutoDataAttribute
{
    public AutoAccountDataAttribute()
    {
        var accountList = Fixture.CreateMany<Account>().ToList();
        Fixture.Register<IRepository<Account>>(() => new FakeAccountRepository(accountList));
    }
}

In the AutoDataAttribute class, Fixture is your main entry point to AutoFixture.

Fixture.CreateMany<Account> will create a sequence of anonymous variables - in this case, accounts. By default, CreateMany will create 3 instances, but this number can be changed using the RepeatCount property of Fixture.

In the Fixture.Register statement, you declare that every time Fixture is asked to create an instance of IRepository<Account>, it will return a new instance of FakeAccountRepository.

Now you can use this custom [AutoAccountData] attribute to decorate any data theory that will test the functionality of the AccountService class. For example, it can be easily tested that a KeyNotFoundException is thrown when trying to get a non-existing account:

C#
[Theory, AutoAccountData]
public void GetNonExistingThrows(AccountService accountService)
{
    Assert.Throws<KeyNotFoundException>(() => accountService.Get(Guid.NewGuid()));
}

The accountService parameter provides a fake account service with a number of fake accounts, so that it only takes one line of code to test that getting an account with a randomly generated (thus non-existing) ID throws a KeyNotFoundException.

Likewise, the Count method can be tested in one line of code:

C#
[Theory, AutoAccountData]
public void CountIsOk(AccountService accountService)
{
    Assert.Equal(3, accountService.Count());
}

The above example clearly demonstrates how AutoFixture dramatically reduces the trivial “yak shaving” of test fixture setup and helps your unit tests become much more declarative, compact and easy to read and understand.

Here is another example where an anonymous account object is automatically generated to test that proper events are raised, when adding a new account to the account repository:

C#
[Theory, AutoAccountData]
public void EventsAreRaisedOnAdd(AccountService accountService, Account account)
{
    var raisedEvents = new List<string>();
    accountService.Adding += (s, e) => { raisedEvents.Add("Adding"); };
    accountService.Added += (s, e) => { raisedEvents.Add("Added"); };

    accountService.Add(account);

    Assert.Equal("Adding", raisedEvents[0]);
    Assert.Equal("Added", raisedEvents[1]);
}

In the sample code, you can find many more unit tests.

Auto-mocking with AutoFixture

Up until now, the unit test samples have been classical state verification unit tests. If you are more of a mockist practitioner preferring behaviour verification unit tests (which, by the way, I am not…) there is another AutoFixture extension available that turns AutoFixture into an auto-mocking container that uses the popular mocking framework Moq to automatically generate test doubles.

On a side note, Martin Fowler describes the differences between state and behaviour verification in his Mocks Aren't Stubs essay.

The [AutoData] attribute, and descendants of this, like the above [AutoAccountData] attribute, only work on concrete types (such as Account). It does not automatically generate objects from abstract classes or interfaces. To enable such functionality, you must add AutoFixture with Auto Mocking using Moq to your test project using the NuGet package manager.

NuGet description of AutoFixture with auto mocking using Moq

Having this in place, you can make a custom extension of the previously described [AutoData] attribute:

C#
public class AutoMoqDataAttribute : AutoDataAttribute
{
    public AutoMoqDataAttribute()
        : base(new Fixture().Customize(new AutoConfiguredMoqCustomization()))
    {
    }
}

Now, you have a truly powerful attribute combining the functionality from the xUnit.net support with auto-mocking capabilities.

When AutoConfiguredMoqCustomization is added to a Fixture instance as done in the above [AutoMoqData] attribute, not only will it behave as an auto-mocking container that automatically generates objects from abstract classes or interfaces. It will also automatically configure all the generated test doubles (mocks) so that the returned values are generated by AutoFixture.

Now, for example, you can test that events are raised whenever adding an entity by requesting mock objects of the abstract BaseService class and the IEntity interface:

C#
[Theory, AutoMoqData]
public void EventsAreRaisedOnAdd(Mock<BaseService<IEntity>> serviceMock, Mock<IEntity> entityMock)
{
    IEntity entity = entityMock.Object;
    BaseService<IEntity> service = serviceMock.Object;
    serviceMock.Setup(s => s.Exists(entity.Id)).Returns(false);
           
    var raisedEvents = new List<string>();
    service.Adding += (s, e) => { raisedEvents.Add("Adding"); };
    service.Added += (s, e) => { raisedEvents.Add("Added"); };

    service.Add(entity);

    serviceMock.Verify(s => s.Exists(entity.Id), Times.Exactly(1));
    Assert.Equal("Adding", raisedEvents[0]);
    Assert.Equal("Added", raisedEvents[1]);
}

So, in this test, there are no concrete implementations (like Account or AccountService) of the abstractions involved. Only mock objects are used.

Notice how elegantly Moq supports Linq expressions when setting up the expectation of the Exists method of the BaseService class:

C#
serviceMock.Setup(s => s.Exists(entity.Id)).Returns(false);

Also, notice that this test verifies that the Exists method is actually called when adding a new entity (so that you cannot add an entity with an existing ID). This is behaviour verification in action:

C#
serviceMock.Verify(s => s.Exists(entity.Id), Times.Exactly(1));

Besides support for Moq, AutoFixture also supports other mocking frameworks like NSubstitute and RhinoMocks.

Practicalities

The sample code is made in Visual Studio 2013 using .NET 4.5.1.

In the sample code, the third party dependencies (xUnit.net and the AutoFixture extensions) are already included, so you don't need to retrieve these dependencies (restore the NuGet packages).

Summary

xUnit.net is a terrific unit testing framework. One of its major advantages, compared to other unit testing frameworks, is that it is extensible. This is what allows tools like AutoFixture to provide the clever and useful extensions to xUnit.net that I have described in this article.

AutoFixture automates the trivial, and often non-relevant, test fixture setup, allowing the test developer to focus on the essentials of each test case. It helps your unit tests become much more declarative and compact and a lot easier to read and understand.

In combination, these two awesome tools make unit testing a true pleasure.

History

  • July 8, 2015 - Upgraded source code to use the most recent versions of xUnit.net (2.0.0) and AutoFixture (3.30.8)
  • September 9, 2015 - Modified broken hyperlink

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Architect
Denmark Denmark
I am a software architect/developer/programmer.

I have a rather pragmatic approach towards programming, but I have realized that it takes a lot of discipline to be agile. I try to practice good craftsmanship and making it work.

Comments and Discussions

 
Praisevery nice Pin
BillW333-Jan-18 9:45
professionalBillW333-Jan-18 9:45 
GeneralRe: very nice Pin
L. Michael4-Jan-18 1:41
L. Michael4-Jan-18 1:41 
QuestionAutoMoqData attribute returns invalidoperationexcetion no data found for Pin
ronanavius26-May-16 5:27
ronanavius26-May-16 5:27 
AnswerRe: AutoMoqData attribute returns invalidoperationexcetion no data found for Pin
L. Michael9-Jun-16 4:40
L. Michael9-Jun-16 4:40 
GeneralMy vote of 5 Pin
rajanet1-Mar-16 17:51
rajanet1-Mar-16 17:51 
GeneralRe: My vote of 5 Pin
L. Michael1-Mar-16 23:16
L. Michael1-Mar-16 23:16 
GeneralMy vote of 5 Pin
Arkitec10-Jul-15 8:01
professionalArkitec10-Jul-15 8:01 
GeneralRe: My vote of 5 Pin
L. Michael1-Mar-16 23:16
L. Michael1-Mar-16 23:16 
QuestionVery interestring Pin
theDiver3-Oct-14 4:40
theDiver3-Oct-14 4:40 
AnswerRe: Very interestring Pin
L. Michael3-Oct-14 12:05
L. Michael3-Oct-14 12:05 
GeneralHiding test setup does not improve readability Pin
nportelli2-Oct-14 7:23
nportelli2-Oct-14 7:23 
GeneralRe: Hiding test setup does not improve readability Pin
L. Michael2-Oct-14 9:48
L. Michael2-Oct-14 9:48 
GeneralRe: Hiding test setup does not improve readability Pin
nportelli2-Oct-14 10:39
nportelli2-Oct-14 10:39 
Generaltest may be easier to read but not as intuitive.. Pin
Nji, Klaus3-Oct-14 13:51
Nji, Klaus3-Oct-14 13:51 
GeneralRe: test may be easier to read but not as intuitive.. Pin
L. Michael3-Oct-14 22:42
L. Michael3-Oct-14 22:42 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.