Afterwood

Back in 1986, Fred Brooks published the seminal essay No Silver Bullet: Essence and Accident in Software Engineering in which he introduced us to the idea that there are two different kinds of complexity which software engineers need to deal with. The first is ‘essential complexity’, which is complexity that is inherent in the problem being solved. Before we can solve the problem, we humans need to understand the problem domain and model it before we can think about how we’re going to represent it in the solution domain. In the essay, Brooks argues that there is little to really help us quickly get into and understand the true essence of the problems we try to solve through software. Yes, there have been some advances but nothing stellar.

The second kind of complexity which Brooks described, he coined ‘accidental complexity’. This exists in the solution domain and is the complexity which exists as a by-product of the processes, tools and technologies we use to solve our customer’s problems. This kind of complexity could be perceived as being a problem of our own making, as the programming languages and tools we use are imperfect and therefore imprecise. These are the silver bullets which the essay is referring to.

Unfortunately, the choice of the term ‘accidental’ became a poor one as it was often interpreted incorrectly. In his follow-up essay almost 10 years later, No Silver Bullet Refired , he describes how it was often treated in the sense of ‘misfortune’ rather than its alternative meaning of ‘incidental’. As such, many rebuttals targeted the apparent incompetence of programmers instead of the imperfections in the tooling, which missed the point of the original essay.

In theory, as we make technological advances we should be reducing the degree of accidental complexity – the ability to represent our solution for the problem in computerised form – and instead spend more time focusing on tackling the essential complexity. The growth of the Agile movement which puts an emphasis on trying to ensure we ‘build the right thing’ before wasting time ‘building the wrong thing, in the right way’ has also meant that the essential complexities are starting to get more of a look-in before we get bogged down in the coding. This is probably not surprising given that the latter part of No Silver Bullet lists the use of ‘incremental development’ and the metaphor to ‘grow, not build’ software as probably the most promising approach to tackling the problem.

No Silver Bullet was added to the Anniversary Edition of his book The Mythical Man-Month: Essays on Software Engineering , which was published 20 years later in 1995. The first essay in that book, and one I’m personally very fond of, is ‘The Tar Pit’. It’s one of the shorter ones, weighing in at just 6 pages, and provides a brief introduction to the rest of book by exploring what systems programming is all about. Along the way he looks at both the ‘joys’ and the ‘woes’ of the craft and suggests that for most people ‘the joys far outweigh the woes’, which I reckon most programmers would find little trouble agreeing with, even if on certain days it feels like the latter might be true.

And so with the lay of the land firmly established I feel comfortable in making my confession – it’s the accidental complexity that I find exciting. That’s right, what floats my particular boat is solving the problems which shouldn’t really exist but do because of those imperfections in the tooling we use. The Internet is awash with advice on how we should get better at understanding our customer’s needs and solve their problems more efficiently but quite frankly that’s just not as interesting to me as trying to dig yourself out of a technical hole which either you (or your predecessor) got you into in the first place.

That doesn’t mean I actively go out of my way to be obtuse or pick technologies that are unsuitable purely for the purposes of self-gratification – on the contrary I still want to be professional and do the best job I can – it’s just I personally often find more pleasure solving the problems in the solution domain than solving the actual customer’s problem.

One example of this would be the war stories that programmers like to share over a pint or two in the bar at a conference. The ones I’m most fond of hearing or reading about (and sharing) are those that involve one person’s struggle against the technology. There are so many wonderful tales about how people have managed to bend, twist and generally contort one tool to make it do something it was never intended to. In An Introduction to General Systems Thinking , Gerry Weinberg suggests that you don’t really know a tool until you’ve abused it at least three times, so perhaps there is a rite of passage where we have to embrace the gnarly problems to truly grasp the very nature of accidental complexity so that we can try to elude it in the future. As an aside, that book was originally published at the same time as The Mythical Man Month (and 10 years before No Silver Bullet ).

My current best efforts to explain these pangs of guilt come from Vivek Singh (@petmongrels) who recently tweeted “Programmer Stack Envy – A belief that work at a lower level in the stack is intellectually more challenging than at one’s own level”. I know that when I leave the ACCU Conference every year, I am in awe of many of the people attending and the things they’ve accomplished, especially as the C++ language grows ever more complex with each new standard. But I wonder how many of those people are also suffering from a form of Stockholm syndrome and are in it largely because it is complex – an enigma to be solved in its own right.

Secretly, I hope there will never be a silver bullet because when that day comes I fear it’s the day the fun goes out of programming.