Rip It Up and Start Again

I have failed to write an editorial. I have notes dotted all around the house, with potential titles or ideas scribbled down. Many are unreadable. Several are so old now, I’ve visited the topics from a different angle in previous excuses for not writing an editorial. Now, if I listened to my own musings in our last Overload (‘Revolution, Restoration and Revival’, [ Buontempo18 ]), I could try to revive some of these. I didn’t. I decided to tidy the house, starting with making a new TODO list, subsequently running out of paper because I remembered several other things that needed to get done. I should throw all my old notes out to give myself a fresh start for a new year. Restoring or reviving something old is one way to go. Other times, the best thing to do is rip it up and start again.

Take a previous talk or an old code base: sometimes you can refactor it slightly, sometimes you can’t. A complete rewrite might be the way to go. In order to refactor, a code base requires some tests, otherwise you are reworking or changing the code instead and potentially changing its behaviour. There might not be conventional unit tests. Sometimes the best you can do is produce some output in a file and use that as a ground truth to ensure any changes introduced don’t cause regressions. Emily Bache ran a 90-minute workshop at the ACCU conference in 2013 on refactoring [ Bache13 ]. The code was a snaky mess of if s and else s, with no tests. However, it did write lots of log messages, so you could tell if it still was functionally identical after making changes. This made it possible to massage the code into a state where you could add a new feature. Watching the number of lines decrease, then the lines, scattered with and s and or s, get shorter was wonderful. Refactoring is a skill I need to work on. I suspect many people don’t really understand what it means. I’ve heard people say they don’t have time to refactor and know they are leaving lots of technical debt behind. Sometimes the same people have gatekeeper-style code reviews. These can work as a place to discuss potential refactoring, and often do. I’ve not often seen a code review out and out reject the proffered solution and demand a complete rewrite. I would hope a team would talk to each other earlier, and realise they are about to walk into a brick wall a long time before a huge patch file needs reviewing. By that point, you have not only lost several days of work, but it’s hard to let go of something you’ve spent a long time on and start over.

Circumstances can force a new start on a project. Sometimes it is a deliberate choice, such as time constraints pushing you into adopting a Plan B. Once or twice, a machine crashed and I lost my work. Another reason to do small commits often. However, I have found that when I do something again from scratch, it is often quicker than my first attempt. I remembered some dead ends I had explored before, or which order to install dependencies in, or an edge case or two that needed a test. I’m not suggesting that destroying your computer is the best way to discover if you are improving. Nonetheless, circumstances sometimes push you to rework or rebuild something. Trying to develop a sense of when you are digging a deep hole for yourself and need to form a Plan B, regardless of outside pressures, is an important skill. Keeping your eye on the prize, by holding in mind what you are trying to achieve, can pull you back from the edge. Being aware of options, so you can unwind your current attempts and try a different method matters. The same goes for trying to mentor or teach others. If one way of explaining things doesn’t work, just saying it louder, more emphatically and waving your hands around probably won’t work. Try diagrams instead. Or some code. Or going for a coffee and moving cups and sugar bowls around. Or get them to explain you what they understand. Shut up and listen.

Now, you might be faced with a challenge that is similar to an algorithm or method you have previously implemented. The promise of object oriented programming suggested code reuse. However, we all know that adding a monster code base as a library to a new project in order to use one or two utility functions is not a sensible plan. At the expense of contradicting my previous cry for a 5p copy-and-paste tax [ Buontempo15 ], it might be wise to copy and paste the functions you need into a new project. You could carve them out into a shared library, of course. But if your needs differ slightly, don’t be shy to look at something you have previously built, and start over, using that for inspiration. In fact, “ Evidence suggests that cut ‘n’ paste is a nearly essential, and nearly universal, practice when new framework applications are constructed, be they for GUI code, ” [ Coplien05 ]. The article goes on to point out that a copy provides a “ safe sandbox ” which cannot disrupt production code “ for a time ”. In some circumstances, this will work like branching and need merging back. For a new project, however, you may have no intention of merging back; instead you are revisiting old code, and re-writing it is a potential learning experience. Any copy can be shallow or deep. Copy-and-paste code is a deep copy or a clone. Two, initially identical, functions or classes exist afterwards. Cut-and-paste, in contrast, simply moves the code. This may happen in order to refer to it from different places, either in a code base, or across code bases. Can you shallow-copy code? I suspect some meta-programming, or something functional would work, but I can’t dream up a sensible use case here. Imagine if you could reference count all your attempts at Fizz Buzz and so on, and garbage collect once in a while, without your computer dying to enforce the code deletion. Various tools for automatic code duplication detection exist. I’ve never used one… well, that’s not strictly speaking true. I think an expensive enterprise tool was being run on a CI box where I worked a while ago that claimed to spot code duplication but I couldn’t make sense of the output it gave. Their docs say, “ A developer has everything at hand to take ownership of the quality of his code. ” Glad to see careless use of pronouns is alive and well. I digress.

Of course, if you try to use old code you might run into issues. An older code base might use a library. Only newer versions may be available, so you can run into compatibility issues. Or in my case, you can’t figure out which version you were using because your machine died and your code repo wasn’t entirely helpful. Beyond libraries, languages themselves evolve and change. C++ has a veritable history of trying to avoid breaking backwards compatibility. Nonetheless, the keyword has been repurposed, and many features, such as auto_ptr have now been deprecated. The new meaning of auto has theoretically stopped some older code working, leaving it needing a rewrite. To be fair, a reviewer pointed out that a couple of the larger companies checked for use in their code bases when the word was originally repurposed and found almost no instances outside compiler test suite. If you are using this, you need to rip it up and start again. Why are interfaces and methods deprecated? “ The deprecation period gives people a chance to change their code before the compiler removes it outright”, according to one stack exchange answer [ StackExchange ]. Where features have been deprecated, you have been warned, but can limp along keeping things as they are. For now. What happens if you are trying to migrate a Python 2 codebase to Python 3? Tools exist to help with the upgrade [ Python ], but they might not catch changes in libraries. Feed our esteemed CVu editor a pint and ask him about numpy and csv files. But only if you have a whole evening free. The next step after deprecation is obsolescence. Whether the old API or technology ever really goes away is another matter.

OK, so we can muse on why deprecation happens. How do you deprecate a method? It is possible to support two APIs in tandem, but that requires careful thought. It can be argued this is better than slapping in a Boolean flag and a new set of inputs to switch the behaviour via the calling code. If you try to switch out a method, you need to be careful not to break existing code. This, in essence, is the Liskov substitution principle. In ‘Kissing SOLID goodbye’, [ Oldwood14 ] Chris Oldwood notes, “ The Liskov Substitution Principle (LSP) is usually ‘explained’ by quoting directly from it. What it boils down to is the proposition that two types are related if one can be used in place of the other and the code still works exactly as before. ” A square is not the same as a rectangle. A stack is not the same as a queue. In contrast, version 2 of a function should still behave the same way as version 1, if called with the same inputs. A naïve and misguided explanation of LSP talks about inheritance. Oldwood’s article points out switching between containers, supporting the same iterator interface, allows a variation point, leaving the behaviour identical. Common interfaces are one way to introduce changes. Feature toggles are another way to migrate to new behaviour. Be warned: these can get out of hand if you don’t retire them quickly.

Various different forces take you to the place of no return, where you do start over. Trying to develop a good sense of when something isn’t working is invaluable. It’s all too easy to keep on digging. That might work out in the end, but might lead to despair. I notice Wikipedia has a disambiguation page for ‘Metanoia’ [ Wikipedia ]. The psychology entry defines it as a psychotic “ breakdown ” followed by a rebuilding or “ healing ”. Sometimes coding feels that way. I know the word from theology, and Wikipedia describes this sense as “ a transformative change of heart ”, carrying the sense of repentance, atonement and conversion. Perhaps you’ve written code you feel the need to repent for? I have. What would you do differently if you revisited this code? The ‘Breakthrough Ideas’ article [ Coplien05 ], referred to earlier, started with a list of questions, one of which was, “I f you could delete any idea, technology, or event from computing history, what would it be? ” What a splendid question. I’d consider voting for an editorial in each Overload , but that might just be me. I’m not sure you can delete an idea. What technology would you delete? OO? VBA? JavaScript? Wizards? Dependency injection frameworks? Anything with the word Enterprise in? What about an event? Recent news about the US joint enterprise defense initiative (JEDI) military cloud request for proposals [ Wired ]. “ This program is truly about increasing the lethality of our department. ” Skynet anyone? It aims to “ inject ” AI into data analysis and provide real-time data during operations. What could possibly go wrong? Some things seem like a bad idea from the outset. Others start well but their time comes. Knowing when that time has come is important. Pick something you don’t like about your approach to programming and stop it. Find a new way forward.

References

[Bache13] Emily Bache, ‘Coding Dojo Challenge-Refactoring’, ACCU conference 2013, available at: https://accu.org/index.php/conferences/accu_conference_2013/accu2013_sessions#coding_dojo_challenge-refactoring

[Buontempo15] Frances Buontempo, ‘Reduce, reuse, recycle’, Overload 130, December 2015: https://accu.org/index.php/articles/2179

[Buontempo18] Frances Buontempo, ‘Revolution, Restoration and Revival’, Overload 148, December 2018, https://accu.org/index.php/journals/2591

[Coplien05] James O. Coplien, Brian Foote, Richard P. Gabriel, Dave Thomas, Cristina Lopes, Brian Marick, Bonnie Nardi, Rob Tow, Andrew Hunt and Glen Vanderburg (2005) ‘Breakthrough Ideas’ in Companion to the 20th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications , ISBN 1-59593-193-7

[Oldwood14] Chris Oldwood, ‘Kissing SOLID goodbye’, Overload 122, August 2014, https://accu.org/index.php/journals/1957

[Python] 2to3: https://docs.python.org/2/library/2to3.html

[StackExchange] ‘Deprecation considered harmful?’: https://softwareengineering.stackexchange.com/questions/83263/deprecation-considered-harmful

[Wikipedia] Metanoia: https://en.wikipedia.org/wiki/Metanoia

[Wired] https://www.wired.com/story/how-pentagons-move-to-cloud-landed-in-mud/