Many of us are aware of the Testing Pyramid. Seb Rose introduces the Testing Iceberg to explain when we should invest effort in making a test readable to non-technical team members.
Almost 10 years ago, I had a conversation with @mattwynne, which led to the hastily sketched piece of paper (Figure 1). The diagram on the left (since redrawn and blogged about by @tooky [Tooke13]) shows the relationship between end-to-end tests and business-readable tests. Not all business-readable tests need to be end-to-end and not all end-to-end tests need to be business readable.
 |
| Figure 1 |
The middle part of the sketch is my attempt to show the relative size of these sets of tests. There should be far more business-readable tests than end-to-end tests. It also shows that most end-to-end tests are business-readable because there are very few situations where purely technical concerns require anything broader than integration tests. The point is, where possible, test the domain model directly and only use end-to-end tests to verify correct ‘wiring up’ of the entire system.
The far right of the sketch attempts to relate the Venn diagram to the well known Testing Pyramid [Vocke18]. Business-readable tests that hit the domain model directly map to the middle section of the pyramid – integration/component tests. Business-readable tests that hit the full stack map to the top of the pyramid. Not shown is where non-business-readable end-to-end tests should map.
At this point I’m going to re-imagine the Testing Pyramid as a Testing Iceberg (Figure 2: another product of conversations with @mattwynne).
 |
| Figure 2 |
Those portions of the iceberg above the waterline are business-readable, while those below are not. As you can see, in this diagram there are examples of all test types both above and below the readability waterline.
Now I can map non-business-readable end-to-end tests to the submerged system test portion of the iceberg, which is very small because most end-to-end tests should be business-readable. Some projects may have specific technical concerns that can only be validated using a fully deployed system, and that are of no interest to business people, but these will be few and far between.
I often get asked how I decide whether a test should be written in a business-readable format (such as Gherkin) rather than a programmer-readable format (such as xUnit). A common anti-pattern is to assume that all end-to-end tests should be business-readable, while all unit tests should be programmer-readable. The Testing Iceberg demonstrates that the question we should actually ask is ‘which tests will benefit from being business readable?’ If your Product Owner or Business Analyst could give useful feedback on the accuracy of the behaviour a test is verifying, then you will get value from writing that test using a business-readable format.
Reference
[Tooke13] Steve Tooke (2013) ‘Cucumber and Full Stack Testing’ published 18 January 2013 at https://tooky.co.uk/cucumber-and-full-stack-testing/
[Vocke18] ‘The Practical Test Pyramid’, published 26 February 2018 at https://martinfowler.com/articles/practical-test-pyramid.html
This article is based on a post published on Seb’s blog on 14 February 2013: http://claysnow.co.uk/the-testing-iceberg/
Seb has been a consultant, coach, designer, analyst and developer for over 40 years. Co-author of the BDD Books series Discovery and Formulation (Leanpub), lead author of The Cucumber for Java Book (Pragmatic Programmers), and contributing author to 97 Things Every Programmer Should Know (O’Reilly).
