Should A Unit Test Contain Only One Assertion?

In a previous post I wrote about questioning grades and a unit test assignment for a class I'm taking this semester. In this post I'm going to write about one of the items mentioned in the feedback I received, which was:

Test methods should contain only one assertion. This way, if a test fails then the program will be more helpful to say exactly which assertion failed (and therefore what the circumstances were that caused the failure).

I completely reject this assertion for two reasons: 1) a well written unit test framework should tell you exactly which assertion failed when there are more than one in a unit test and 2) having a single assertion per test makes the tests more fragile and increases the likelihood that the tests themselves will be flawed.

The first reason is fairly obvious. Since all runtime environments provide stack traces on exceptions, there's no reason why a unit testing framework can't display one on a failed assertion. So the framework can easily tell you which assertion within a single test failed. Therefore there's no reason to hav ea single assertion in a test since the stack trace can take you directly to the failure. And, additionally, mode IDEs will let you jump directly to the line so you can verify the failure and fix the cause.

So a single assertion per test method is not more helpful in finding the cause of a failure.

The second reason is the more serious of the two since I believe it leads to bad coding practices. That idea that unit tests should be limited to a single assertion per test assumes that a unit test is only verifying one specific post-condition. Take, for example, an object cloning operation:

@Test

public void testClone() {

    Farkle source = new Farkle();

    Farkle result = f1.clone();

    assertNotNull(result);

    assertEquals(source, result);

}

In this example there are two assertions being performed: the first one ensures that an object was returned by the clone() method and the second ensures that the returned object fulfills the public contract for the clone() method; i.e., the object returned is identical to the original.

Now, if we were to follow the "one assertion per test" tenet then the above would have to be split into two unit tests:

public void testCloneReturnsAndObject()

public void testCloneReturnsEqualObject()

We've now doubled the number of test methods. However, we haven't done anything more than that. Sure, it can be argued that we have made the tests more specific and that that is itself a good thing. But we are now also required to setup the same test preconditions twice since both tests have to instantiate the same source and run the same method to produce the same post-condition only to test different aspects separately. This violates the concept of DRY and as such is a bad practice.

"Well, why not move the initialization into your setUp() method?"

The answer is this: setUp() (a method specific to JUnit, but each testing framework has its own analog) isn't supposed to setup every single test's preconditions. It's role is to setup the general preconditions for the entire test case itself..

To make setUp do the work for every single test would make it very inefficient. Since it is run before every one of the unit test in the class (in JUnit the setUp() method, or any method annotated with @Before, is called before every single unit test method is invoked). So that's a LOT of wasted set up work if we were to move all preconditions out of the tests themselves.

Additionally, by moving the preconditions out of the test methods we hide what the expected state is for those tests. Now you would have to go to the setup method to see details that should be right there in the test, making it harder than necessary to debug when code stops passing tests. Something easily avoided by keeping the precondition code in the test itself.

So now that I've defined why the preconditions need to stay in the test method itself, that leads us to the final reason why a single assertion per test method is a Bad Thing (tm): the test code itself could easily become invalid.

Imagine a slightly more complicated test, such as one that verifies five different aspects of the result of a method call.This also violates the DRY principle since we're doing the same setup in five different places. We also run the risk of the tests themselves changing as, with refactoring, one test will get updated while another may not. And down the road those differences can result in false negatives or, worse yet, false positives.

Instead, if we keep only the number of assertions as necessary in a unit test, we make the test themselves more expressive, by showing expected preconditions, and more targeted, by checking for expected outcomes.

Requiring tests to contain only a single assertion is a bad practice that leads to writing more code than necessary and either repeating yourself or else writing inefficient, and potentially ineffective, tests.