Afterwood

When I was a boy, I got an illustrated encyclopaedia as a present one Christmas or birthday. I wasn’t into reading fiction and loved flicking through the book looking at the various cut-away drawings of machines – both old and new – as I was fascinated by how things worked. The one section of the book I remember most clearly though was a picture of a man in a running race with a tortoise. The text described how it was impossible for the man to ever beat the tortoise, if the tortoise had a head start, because when the man reaches the point where the tortoise stood, the tortoise will have moved on. While the book probably referred to the puzzle by its more well-known name of Achilles and the Tortoise, I never managed to remember that. It probably even mentioned this is just one of Zeno’s many paradoxes, as I discovered some decades later via Wikipedia.

Being a child, I wasn’t capable of unravelling the paradox and pointing out where Zeno ‘had cheated’ but I knew he must have because empirically I saw the outcome disproved every day on the school field at playtime. And yet the logic in the simple statement made perfect sense. Despite reading the various rebuttals numerous times over the years, I’m still not entirely sure I fully understand where the falsehood lies in theory, but I do know that it does in practice.

Over the last couple of years, the pandemic has made me aware of some other interesting statistical anomalies as a variety of people have tried to make sense of the ever growing body of data around the effects of the virus so that models can be built, hypotheses formulated, and (ideally) policy implemented to minimise its effects. One anomaly that seemed to crop up repeatedly in the early days of the pandemic was Simpson’s Paradox. In an explanation of its effects, the particular example that most struck a chord was a plot showing that ‘in general’ more exercise caused an increase in cholesterol, which clearly goes against medical advice. For those of you like me not well versed in statistics, the illusion here is that what you’re seeing is just the natural rise in cholesterol as we get older. If you break the plot down into clusters based on age, you see that within an age group cholesterol does indeed go down with exercise. The devil, as they say, is in the details.

Another popular mathematical puzzle which crops up regularly is the Birthday Paradox, which has serious applications in cryptography, not just as a parlour trick. The problem is often stated as: “How many people need to be in a room for there to be a better than 50% chance that any two share a birthday?” The answer is 23, which is surprising to a lot of people; hence it’s commonly described as a paradox because the answer is counterintuitive, even though with the right level of maths ability you can ‘easily’ prove it. Somewhat ironically, according to the Wikipedia page, the reason it’s discoverer Harold Davenport didn’t publish his findings was because he couldn’t believe it hadn’t already been published.

I think we programmers tend to think of ourselves as a pretty logical bunch. The act of programming requires us to be incredibly precise in our instructions to the computer, a device which makes most pedants appear quite liberal. Our job involves reasoning about problems and, where necessary, turning this into code which will ultimately be consumed by a gazillion logic gates. It’s logic all the way down. And yet my take-away from Achilles vs Tortoise is that what I think might be logical may in fact be flawed because I can’t obviously see a counter-argument. In our eagerness to get something working or a bug fixed we can fall into the trap of formulating a hypothesis that can be proved correct when we should be finding ways to disprove it, or in the very least putting the scaffolding in place to make that process easier. In some respects, the practice of TDD borrows from the ideas of falsifiability as it puts a focus on the testability angle of our code (hypothesis). By making the code testable, we make it easier for ourselves to try and conjure up reasons why the code might not work correctly, and ultimately express those scenarios too, in the form of executable code. Much as I’d like to believe I can follow Sir Tony Hoare’s advice and “make the program so simple, there are obviously no errors” I’m also aware of the assumptions that can eventually lead to our undoing.

Larry Wall famously described the three great virtues of a programmer as laziness, impatience, and hubris. I’ve always found the last a curious choice as hubris is commonly used as a pejorative, although I understand it can be seen as a positive force in some circumstances. I wonder how many others misinterpret that quote leading to a lack of humility? In contrast I find fallibility is a more useful position to take.

I once got involved in a support query involving corruption of a cache file. The programmer who wrote the file handling code reached a conclusion that there must be a bug in the Windows CopyFile API rather than doubt his own code. While I would never rule that out entirely, there are many reasons why I think it’s improbable and I’d be looking far closer to home before even contemplating such an idea. A quick trawl through the parent process’s logs and I pointed out that the child process was now being terminated for taking too long to start-up, ultimately caused by an increase in the cache file’s size and I/O load on the remote server. This felt more plausible and, more importantly, easier to validate.

Of all the paradoxes in Wikipedia’s long list, the one I’m finding easiest to relate to as I get older and more experienced is Socrates’ apocryphal saying “I know that I know nothing”. Clearly I do know something, lots in fact, but the pace with which our industry moves it can feel like technology is the tortoise and I’m Achilles desperately trying to keep up.

Chris Oldwood is a freelance programmer who started out as a bedroom coder in the 80s writing assembler on 8-bit micros. These days it’s enterprise grade technology from plush corporate offices the comfort of his breakfast bar. He has resumed commentating on the Godmanchester duck race but continues to be easily distracted by messages.