Documentation as Code

I remember being blown away when I first saw Cucumber roughly a decade ago. It was like writing documentation, and then executing the documentation against your actual codebase. Wow!

A decade later I firmly believe that automated tests should be thought of first as documentation that can also be executed, even just your plain old unit tests.

We’ve all seen codebases where an initial enthusiasm for unit testing slowly erodes as the tedium of maintaining low-value tests doesn’t seem worth it anymore. Tests are commented out when they break, and eventually new tests are no longer added.

And we developers hate writing documentation! It’s tedious, docs get out of sync with updates to the codebase, and nobody reads it.

Well, I like to think of automated tests as a chance to write useful documentation for once.

And that’s why I believe the #1 most important quality of a test is readability!

As Roy Osherove says in The Art of Unit Testing:

Readability is so important that, without it, the tests we write are almost meaningless.

Tests are stories we tell the next generation of programmers on a project.

So rather than piling tests onto a codebase just to see a code coverage metric go up, let’s ask: Have I made the codebase easier to understand today?

I Prefer to Read the Classics

 

Programs must be written for people to read, and only incidentally for machines to execute.

- Abelson & Sussman, Structure and Interpretation of Computer Programs

I like to occasionally revisit Martin Fowler's article Mocks Aren’t Stubs whenever I’ve been thinking a lot about unit testing. I tend to pick up on nuances that I didn’t pick up on previously.

In the article, Fowler breaks down what he calls the Classicist vs. Mockist schools of unit testing. He describes what he sees as the advantages and disadvantages of each approach in a fairly unbiased way.

In my most recent pass through the article, I noticed that he leaves out what I see as an important facet of this divide in approaches: readability of tests.

Here’s an example of the Classicist style of unit testing that Fowler uses in the article:

public void testOrderIsFilledIfEnoughInWarehouse() {
    Order order = new Order(TALISKER, 50);
    order.fill(warehouse);
    assertTrue(order.isFilled());
    assertEquals(0, warehouse.getInventory(TALISKER));}

 

And here’s a functionally similar test he provides in the Mockist style of unit testing, which I understand may not be at the cutting edge of Mockist style, but I think illustrates the problem for me:

public void testFillingRemovesInventoryIfInStock() {
    //setup - data
    Order order = new Order(TALISKER, 50);
    Mock warehouseMock = new Mock(Warehouse.class);
    
    //setup - expectations
    warehouseMock.expects(once()).method("hasInventory")
      .with(eq(TALISKER),eq(50))
      .will(returnValue(true));
    warehouseMock.expects(once()).method("remove")
      .with(eq(TALISKER), eq(50))
      .after("hasInventory");

    //exercise
    order.fill((Warehouse) warehouseMock.proxy());
    
    //verify
    warehouseMock.verify();
    assertTrue(order.isFilled());
}

In my opinion, Mockist-style unit tests are much harder to read than Classic tests. And readability is king. If you scan a test and think to yourself I have no idea what this test is supposed to prove, then that’s a bad test.

And to be frank, I sometimes feel that mock-heavy tests are more a demonstration of Look what objects can do! than a practical means of gaining confidence in one’s software. Each new mocking framework tries every syntactic trick it can think of to mitigate the fact that behavior-based verification is inherently an awkward concept.

If the primary test of a test is readability, then the Classicists have it.

TDD and Double Entry Bookkeeping

I want you to think of TDD the way accountants think of dual entry bookkeeping. I want you to consider it as an essential part of your profession. I don't want you to think of it as optional. I don't want you to think of it as a luxury. I want you to think of it as absolutely necessary to your professional and personal self respect.

- Robert C. Martin

TDD as Part of Your Marketing Pitch

I just finished listening to my coworker, Chris Woodruff, interviewing Josh Holmes at CodeMash 2008. It's a great interview (if a bit too long, honestly).

One of the things that really grabbed my attention was when Josh mentioned an acquaintance who includes an absolute commitment to test-driven development as part of the marketing message from his consulting company. This acquaintance proudly makes known that all builds at his company automatically fail if the test coverage is less than 100%.

I've heard that before (actually, from a local competitor of ours).

I can't help but think: wouldn't it be fantastic if we could include an absolute commitment to quality, backed by TDD, as part of our marketing message? Competitive advantage, anyone?