The legacy of misplaced Testing

Recently I’ve been exposed to a number of projects that have been going on for a few years. As you’d expect they are at a stage where the cost of change is phenomenal. The codebase is large, convoluted and very difficult to understand. However, there are a lot of tests both at the system and unit level. A sigh of relief – if the code has a lot of tests then at least that would make refactoring easier. The reality is very different – the tests are even more complicated then the code and significantly add to the cost of change.

Not only we have convoluted unit tests we have a very large set of system tests that exercise lots of variations on the main flows. These are very expensive tests because they require a lot of test data to be setup and go through the full flow in order to verify a very small part in that flow. You can see that as the code base became more convoluted and dependencies became difficult to mock, the developers started  relying more and more on the system tests which proved comparatively easier to setup by copying the setup data from previous tests. If you have hundreds of these and no one knows exactly how many of the main flows/scenario are exercised then what good are these system tests?

How do you turn this tide? One approach is to start moving these variations into unit tests in order to reduce the system test suite into a very small set of targeted tests that are testing the main business scenarios. Granted you’ll have to spend effort on untangling the dependencies. You can adopt an automated testing framework (e.g. JBehave, Cucumber etc.) so the system and unit tests are defined in the language of the business. The units under test become more coarse grained to satisfy these scenarios. This may allow swathes of complicated unit tests to be replaced – don’t be afraid to remove them. You must be horrified – remove tests! – well if they are not improving our confidence in refactoring then one can argue that they are no longer fit for purpose.

I am in no way saying that we stop doing TDD/BDD. Lack of TDD/BDD is most likely the reason we get ourselves into this situation in the first place. This is more an approach to lessen the pain of misplaced and substandard tests – to allow us to gradually turn the tide on the cost of change.


Naming your Unit Tests

Often I see Unit Tests with the test methods that have the same name as the method under test prefixed with the word “test” e.g. testSubmitApplication. This provides no extra information on which “flow” of the mothod is being tested. Other test method names provide a bit more information by suffixing the nature of the test e.g. testSubmitApplicationWithInvalidCriteria. It better but not much better. A number of IDEs actually allow the developer to generate test method names based on the class under test which in my opinion defeats the object.

Unit test methods should be named in such as way that they provide a clear description of the test. In my opinion the prefix “test” is redundent and should never appear in your test method names unless it is part of the domain vocabulary. For example AppicationSubmittedWithInvalidCriteriaMustRaiseException* is more informative then testSubmitApplication. Providing a more descriptive name also serves to keep a clear focus on the flow under test and leads the devloper to create a test method per flow.

*Please Note that the example method name is a simplification. I would consider the term “InvalidCriteria” a bit too high-level for a real unit test. It should be more specific such as AppicationSubmittedWithNoSurnameMustRaiseException.

Running NUnit in Visual Studio 2008

Follow the step-by-step instructions to run NUnit in Visual Studio 2008:

  1. Download and install NUnit
  2. Add a new Test project to Visual Studio 2008 Solution
  3. In Windows Explorer, create a folder called “lib” in the solution direcotry
  4. Copy the nunit.framework.dll from NUnit installation and add to lib folder
  5. In Visual Studio right-click on the solution and click “Add -> New Solution Folder
  6. Add Existing Item to the solution folder pointing to the dll
  7. In the Test project add the framework dll and a reference (right click on References and click Add Reference)
  8. You can now start creating tests. Have a look at the quick start guide on how to write tests.
  9. Build the test project then navigate to the project’s debug folder in Windows Expolrer and double-click on the project dll. This should startup the NUnit console to run the tests you have created. If you have ReSharper then you can run the test from Visual Studio.