Kevin Dostalek
Vice President, Solutions Delivery & Innovation
This year has seen a huge advancement in the technology available to development teams for the automation of unit tests. This has been primarily driven by an upswing in organizational adoption of Agile development methodologies which by and large demand the use of automated test suites to insure high levels of quality in the face of change. Test Driven Development (TDD) has taken this to the next level as it actually requires the tests be written prior to the code. This article explores many of the traditional challenges surrounding automated unit testing and highlights some of the new tools available to overcome them.
What is an Automated Unit Test?
An automated unit test is simply a piece of code that exercises another piece of code in an attempt to assert that it is working as expected. The benefits of testing in this manner include:
• Providing a regression test library so that upon any change to the code-base (and certainly upon the checking-in of new code) you can prove that the changes have not “broken” any other previously expected behavior.
• Allow for much quicker integration testing because the build server (continuously integrating) can run the test suites automatically.
• Provide a documentation source that provides the elusive answer to the question: “what was the developer intent”.
• In the case of TDD, unit tests provide a structured way to attack complex problems in smaller (presumably easier) increments.
What do I test?
What to test is a highly controversial topic and there are many lines of delineation that can be drawn (only public methods, complete code coverage, etc.) The direction I generally give my developers is to write tests for anything that has a reasonable likelihood of breaking if we change something. By running code-coverage reports in Visual Studio 2008 you can get a feel for what areas of the code are being exercised or not and write additional tests if you find potential trouble spots that aren’t being covered. Testing private methods has traditionally been difficult without resorting to “tricks” that often are more complicated than the code under testing. However, new IDE’s such as Visual Studio 2008 allow you to simply pick those methods from a wizard and it produces a “Private Accessor” class automatically that encapsulates the reflection magic, effectively abstracting away much of the complexity.
How do I write variant tests?
Writing tests to test all possible runtime conditions is very time consuming and sometimes not practical. However, many testing frameworks including nUnit, MSTest (Visual Studio), and MBUnit, support the writing of data driven tests in which you can provide a list of variables from a data source and have the test run multiple times with different sets of variables. A new tool from Microsoft Research called Pex takes this one step further by providing automated exploratory testing. The way this works is the developer writes a parameterized unit test and then Pex performs static analysis and runtime monitoring on both the parameterized unit test as well as the code being exercised to generate a suite of standard unit tests with very high code coverage.
What about dependencies?
Often we run into testability issues because our code relies on dependencies and it becomes tricky to isolate the code under test. The recent increase in adoption of architectural patterns such as MVC (see ASP.Net MVC, MonoRail, ProMesh, etc.) and MVP (see WCSF) help minimize this problem by the separation of concerns and the use of dependency injection containers (see Unity, ObjectBuilder, Castle Windsor, Spring.Net, etc.). Since they all allow dependencies to be injected at runtime you can inject the real dependencies in production and mock objects when running tests. Speaking of mock objects, while I still like writing my own, the mock object frameworks that greatly simplify this task have come a long way as well. My two favorites (which take very different approaches) are TypeMock and Moq. NMock and RhinoMock are also great choices that have quite a following.
How do I test in a real web context?
Here as well there are some great new tools that can be used to write true web-context (not mock) driven tests. The first is the Visual Studio Team System Test Edition which supports writing web tests directly in the Visual Studio IDE. The ability to leverage these same tests for load testing is an added benefit as well as easy integration with TeamBuild if you are using that for continuous integration. An even better tool I’ve found is Selenium which lets you compose your tests visually via a FireFox add-on (asserts and all) and then produces code in your target language (e.g. C#) that you can paste directly into your test framework such as nUnit or MSTest.
What about business driven acceptance tests?
Usually your user acceptance tests will be in the form of system testing and probably not great candidates for automated testing. However, there are many situations where the business drives some complex logic or calculations that really should be tested within a unit test. Since the previously mentioned unit test frameworks involve coding, we can’t exactly expect business users to author these tests. However, frameworks like Fit have evolved into viable tools that allow customers and developers to learn what their software should do and what it does do by providing interfaces that allow customers to feed parameters and expected outputs directly into developer provided unit tests and see immediately the test results based on expectations.
In Conclusion
There are many tools available today that take much of the tedium out of creating automated unit test suites. With many barriers removed, there is really no reason to not increase your software development quality and efficiency by leveraging the practice of automated unit testing. An excellent resource for those trying to create an organizational standard can be found at SSW.