TDD Discussions and Disagreements

In recent weeks, a TDD debate has been raging (again) in the blog-o-sphere and on Twitter. A lot of big names have been making bold statements and setting out arguments, of both the carefully constructed and the rhetorically inflammatory variety. I’m not going to revisit those arguments – go read the relevant posts, which I have collected in a handy timeline at the end of this post.

Everyone is right

Instead of joining in the argument, I want to consider a conciliatory post by Cory House entitled ‘The TDD Divide: Everyone is right’. He proposes an explanation for these diametrically opposed views, based upon where you are in the software development eco-system:

Software ‘coaches’ like Uncle Bob believe strongly in TDD and software craftsmanship because that’s their business. Software salespeople like Joel Spolsky, Jeff Atwood, and DHH believe in pragmatism and ‘good enough’ because their goal isn’t perfection. It’s profit.

This is a helpful observation to make. We work in different contexts and these affect our behaviour and colour our perceptions. But I don’t believe this is the root cause of the disagreement. So what is?

How skilled are you?

In Japanese martial arts they follow an age old tradition known as Shu Ha Ri, which is a concept that describes the stages of learning to mastery. This roughly translates as “ first learn, then detach, and finally transcend ”. (I don’t want to overload you with Japanese philosophy, but if you are interested, please take a look at Endo Shihan’s short explanation.)

This approach has been confirmed, and expanded on, in modern times by research conducted by Stuart and Hubert Dreyfus, which led to a paper published in 1980. There’s a lot of detail in their paper, but Figure 1 shows the main thrust of their findings.

Figure 1

For me, the important point is that novices follow rules and ‘don’t know what they don’t know’. They need to be given unambiguous instructions while they learn what’s important.

At the other end of the spectrum, experts use their intuition and metacognition (an awareness and understanding of one’s own thought processes). They often can’t accurately describe how or why they arrived at a particular decision.

Who are they talking to?

When presenting information it’s important to understand who your audience is. Material that’s suitable for novices is often not useful for experts. Material for experts can be dangerous in the hands of novices.

Some of the contributors to the TDD debate do try to indicate who they are writing for, but most don’t. And even when you do say something like “ for experts only ” you’re leaving it to the reader to decide if they are an expert or not, which is a decision that we are poorly equipped to make. In fact, the Dunning-Kruger effect shows that people who are in the early stages of acquiring a skill frequently overestimate their competence, while those who are very skilled tend to underestimate it.

The internet is an open resource. No matter what sort of health warning you put on your blog posts, they will inevitably be read by people of differing skill levels. And therein lies the problem. The bloggers have an implicit idea of who their audience is – and it might not be you.

The people who warn against TDD usually come from environments where the team is working at competent level or above and that’s who they are talking to. If you respond to these posts by thinking “ Oh cool. This person is saying that I don’t have to do this difficult thing ” then you’re probably a novice. Carry on practising until it becomes easy and then you’ll know why they didn’t find it useful.

Those who promote TDD often work with teams that are less skilled and they have seen the benefits that derive from acquiring these skills. If TDD is something you’ve already become comfortable with, then you’ll know why you’ve chosen not to use it any more and it won’t be “ because it was difficult ”.

To TDD or not to TDD. That is not the question.

TDD is a technique that has costs and benefits. Whether it is right for you and your colleagues depends on your team’s context – the domain, the skill level, the schedule, the risk. Like any technique, it’s no use if you do it badly. Like any technique, it can neither help nor hinder if you don’t apply it. If you want to get good at TDD you have got to practice it.

You will need to get good at writing robust unit tests that exercise the behaviour not the implementation. You will find yourself getting better at creating designs that are made up of smaller, cohesive, decoupled components. You will have an automated suite of tests that give you fast feedback, the confidence to refactor and protection from regressions. You will have executable documentation that doesn’t rot over time.

If your development process doesn’t deliver these benefits, you’ll have inevitably felt the pain caused by long debugging sessions, unexpected regressions, poorly structured code and stale (or missing) documentation. There will be parts of the codebase where you fear to tread.

You can acquire unit testing and design skills without practicing TDD, but many people find that the structure of TDD really helps keep focus. Once you get good, you’ll be competent (or better) in all these skills and well placed to decide whether the cost of TDD outweighs the benefits or not. The skills you’ve picked up on the way will be invaluable whichever choice you make.

But if you’ve never tried TDD, or you’ve never practised enough to be comfortable with it, then you’re still a novice. My advice is to keep on practising – it’ll be worth it.

Teaching TDD (TTDD)

Another flurry of discussion about TDD was sparked off by a recent post from Justin Searls [ Searls ]. In it he lists a number of failures that range from “ Encouraging costly Extract refactors ” to “ Making a mess with mocks ” all of which distract attention from the concept that “ TDD’s primary benefit is to improve the design of our code ”. He concludes by suggesting that once you have written a failing test, rather than get-to-green (see Figure 2) in the simplest way possible you should “ intentionally defer writing any implementation logic! Instead, break down the problem by dreaming up all of the objects you wish you had at your disposal ”. In essence, design the elements of the solution while the first test is still red.

Figure 2

It’s an interesting post that raises a number of issues, but for me its value lies chiefly in opening the subject up for debate. The introduction is particularly pertinent – just setting a class a bundle of katas to do does not, of itself, encourage learning. The pains experienced while doing the exercise need to be teased out, discussed and have alternative approaches described. If you don’t hear the penny drop, then it hasn’t dropped.

Pitching in with characteristic vigour and brimstone came Uncle Bob with a robust rebuttal containing both heat and light (though some have been put off by the heat and never got to the light). Bob makes some good points regarding the fallacy of writing tests around extracted classes, the tool support for extract refactoring and the central place of refactoring in the Red-Green-Refactor cycle.

By the conclusion, however, Bob has switched tack. He states that while refactorings are cheap within architectural boundaries, they are expensive across them. Whether he’s right or wrong on this point ¹ is of no concern because now he’s addressing the wrong question. The question at hand is “ how is it best to teach TDD? ” and it was taken up in a short Twitter exchange between me, Kevlin Henney and Mike Long.

Mike, being hard-core, says that he teaches TDD by “ start[ing] with writing the test framework. Start from assert and up. ” There’s much more to discover about this approach, and it’s certainly reminiscent of how Kent Beck learns a new language (by reimplementing xUnit).

Kevlin, in an e-mail, is more discursive. He works with a mixture of prepared material, discussions, instructor-led demonstrations and learning exercises. There are katas in the material, but there’s lots of interaction to draw out the key points and really get them to stick.

In my TDD training, I start by focusing on fundamental unit testing practices. The key word here is ‘fundamental’. TDD is unit testing++, so you need to have a firm grasp of what a good unit test looks like. We work through a series of simple bank account examples (in pairs, using Cyber-Dojo, of course [ CyberDojo ]) that bring out the 6 essential properties of unit testing [ Rose12 ]. I then use an example (based on an idea of Rob Chatley [ Chatley ]) to introduce test doubles before moving into one of my own legacy code exercises. In-between I use a couple of the katas that ship with Cyber-Dojo – usually LcdDigits and PrintDiamond – to get a more varied domain experience.

There are lots of ways to teach TDD and Justin Searls has certainly identified one way of doing it sub-optimally. I have to disagree with his conclusion that the fix for this is to defer implementation till after the solution design is fully sketched out. At the opposite end of the spectrum is Keith Braithwaite‘s “ TDD as if you meant it ” [ Braithwaite09 ] which is an exercise you can (and should) try at home.

In my opinion, the success of any training is dependent on the trainer – the material is of secondary importance. So if you decide that some TDD training is for you, remember to think about who is training you, not just how long the course is and how much it costs.

References

[Braithwaite09] Keith Braithwaite, (2009) ‘Thought-provoking TDD exercise at the Software Craftmanship conference’, February 2009. http://gojko.net/2009/02/27/thought-provoking-tdd-exercise-at-the-software-craftsmanship-conference/

[Chatley] Robert Chatley http://chatley.com/

[CyberDojo] Cyber-dojo website: cyber-dojo.org

[Rose12] ‘Bad Test, Good Test’, Seb Rose, available on slideshare, http://www.slideshare.net/sebrose/bad-test-good-test

[Searls] Justin Searl’s blog: http://blog.testdouble.com/

Selected posts and tweets

September 30, 2008 – Kent Beck http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests/153565#153565

Unknown date, 2009 – J.B. Rainsberger http://www.jbrains.ca/permalink/how-test-driven-development-works-and-more

October 6, 2011 – Ian Cooper http://codebetter.com/iancooper/2011/10/06/avoid-testing-implementation-details-test-behaviours/

May 1, 2013 – Steve Fenton http://www.stevefenton.co.uk/Content/Blog/Date/201305/Blog/My-Unit-Testing-Epiphany/

May 13, 2013 – Steve Fenton http://www.stevefenton.co.uk/Content/Blog/Date/201305/Blog/My-Unit-Testing-Epiphany-Continued/

July 15, 2013 – Philip Ledgerwood http://thecuttingledge.com/?cat=5

January 13, 2012 – Dan North https://twitter.com/tastapod/status/157633913009864704

June 12, 2013 – Ian Cooper http://vimeo.com/68375232

January 25, 2014 – Justin Searls http://blog.testdouble.com/posts/2014-01-25-the-failures-of-intro-to-tdd.html

January 27, 2014 – Uncle Bob http://blog.8thlight.com/uncle-bob/2014/01/27/TheChickenOrTheRoad.html

February 25, 2014 – Santiago Basulto https://medium.com/tech-talk/e810d9b4fb02

April 23, 2014 – David Heinemeier Hansson http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

April 25, 2014 – Uncle Bob http://blog.8thlight.com/uncle-bob/2014/04/25/MonogamousTDD.html

April 29, 2014 – Kent Beck https://www.facebook.com/notes/kent-beck/rip-tdd/750840194948847

April 29, 2014 – David Heinemeier Hansson http://david.heinemeierhansson.com/2014/test-induced-design-damage.html

April 30, 2014 – Gary Bernhardt https://www.destroyallsoftware.com/blog/2014/tdd-straw-men-and-rhetoric

April 30, 2014 – Uncle Bob http://blog.8thlight.com/uncle-bob/2014/04/30/When-tdd-does-not-work.html

April 30, 2014 – Tom Stuart http://codon.com/how-testability-can-help

May 1, 2014 – Uncle Bob http://blog.8thlight.com/uncle-bob/2014/05/01/Design-Damage.html

May 1, 2014 – Cory House http://www.bitnative.com/2014/05/01/the-tdd-divide/

Acknowledgements

The diagram in Figure 2 was provided by Nat Pryce.

Previously published

This article was previously published at: http://claysnow.co.uk/to-tdd-or-not-to-tdd/ and http://claysnow.co.uk/teaching-tdd-ttdd/

• Grady Booch once said: “All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.” Steve Tooke pointed me to an old post by Nat Pryce, which hints at a different trade off between change and cost. After all, when was the last time you were perfectly happy with an ‘architectural’ decision that was made more than a week ago by somebody else?