đżď¸ Murder Your Darlings - The IPT Newsletter
Originally published 22nd July 2024
Iâm going to let the dust settle on the biggest IT incident ever before writing about canaries and âtrust but verifyâ next weekâbut if you really want my detailed take on minimising the blast radius, check out my three-parter on LinkedIn (first part, second, third) or just pay attention to the final sentence:
In every airport, bank, and hospital, and in similar organisations running critical services, whether or not they were hit by Friday's disaster, Monday's first task should be reviewing software supply chains to find and remove any path to production changes that isn't double-checked.
Why does a mirror reverse your left and right sides, but not exchange your head and your feet? This is actually a subtle illusion caused by the symmetry of the human body: youâre imagining that you could step into the mirror, turn around, and become what you see on the other side of the pane, but in fact whatâs going on is something else entirely. You canât become the image you see in the glass because it inhabits a completely different world.
Software design and delivery suffers from a similar distortion. A âcomputerâ used to be a human, and we still treat our silicon friends as if they were mirror images of ourselves, incredibly fast but fundamentally performing just the same steps as we do. So itâs natural for us to think of errors or deficiencies in a program as easily correctable; all we have to do is tell the machine to keep better track of which task to do first, or undo what we just did, exactly as we would direct a person. But every engineer fears the change request that starts with âJustâ or âSimplyâ, because whatâs demanded is inevitably not at all straightforward. The computer thatâs displaying these words to you is constantly doing work it throws away, storing information it guesses youâll want, and interrupting itself to do housekeeping youâre completely unaware of. Hidden complexities like these are what make instructing a machine so incredibly delicate and frustratingly slowâand those same subtleties have a terrible habit of magnifying tiny mistakes into billion-dollar global catastrophes.
This is why I regularly emphasise the importance of defining your negative space, the features and challenges youâre not going to take on with your software, and relentlessly enforcing those decisions through constant triage. Just like a nurse on the scene of a disaster, you need to dispassionately assess which tasks and bug fixes are worth saving and which are too far gone to be admitted to the development pipeline. And you need your engineers to be intimately involved in that process, since they are best placed to understand the âmirror worldâ where the hard becomes easy and the easy impossible. Sometimes itâs even best just to delete your backlog and start over.
I was reminded of just how hard this advice can be to follow when I met with an executive at a well-known global charity this week. His employees came to the organisation to serve a noble purpose, not to enrich themselves or shareholders, and so it is particularly difficult for these committed team members to say ânoâ to changes that would, after all, build communities and save babies. Keeping morale high and fighting burnout is a constant challenge when your team know that decisions they make may have such visceral impacts. But itâs even more important in the âpurpose-drivenâ world to screen relentlessly to preserve the possible and discard the undoable, or the strain to do too much may doom the overall missionâsee for example the cautionary tale of One Laptop Per Child. Whether your purpose is profit or philanthropy, taking concrete steps to build trust and psychological safety will help your team muster the courage to âmurder your darlingsâ.
This first appeared in my weekly Insanely Profitable Tech Newsletter which is received as part of the Squirrel Squadron every Monday, and was originally posted on 22nd July 2024. To get my provocative thoughts and tips direct to your inbox first, sign up here: https://squirrelsquadron.com/
