🐿️ Your Software Is Wrong - The Insanely Profitable Tech Newsletter
Originally published 12th May 2025
Measure Twice, Compute Once
The latest thing in 17th-century “artificial intelligence” was an 80-page book full of numbers in tiny type compiled by Adriaan Vlacq, extending earlier work by Henry Briggs. Computers (the kind made of flesh and blood) could do multiplication and division the way you and I learnt in school, but it was slow and error-prone. The newfangled logarithms crammed onto the pages transmuted these complex operations into much simpler addition and subtraction instead, with an effect on speed and accuracy akin to getting a new phone or CPU chip for us today. But there was a subtle flaw in the new technology: many of the figures in Vlacq’s tome were just plain wrong.
Whether due to miscalculation or mistyping, almost every page of the Arithmetica Logarithmica had a mistake of some kind in its dense rows of figures, requiring vigilance and double-checking from readers. And what’s incredible is that for the next two centuries, many of these slip-ups were dutifully copied into new editions and extensions of Vlacq and Briggs. Publishers tried to keep up by including lists of errata, but as late as 1872 mathematicians were still finding mistakes.
The 0.6% error rate wasn’t due to lack of care or diligence: the authors used a wide variety of cross-checking techniques to compensate for their own human frailty. Like bookkeepers comparing credits to debits and running reconciliation reports, they calculated the same results over and over using different methods to flush out and fix errors–but their best efforts still fell short, and caveat lector remained the wisest rule to follow until the advent of the digital computer.
The surprising thing is that the tragedy of the logarithms is just as relevant in the 21st century as it ever was, albeit at a much larger scale. Our silicon friends multiply and divide with far greater precision than even the best tables would allow, but subtle mistakes in chip design can still lead to wrong answers in the fourth decimal place and satellite designers worry about computations being thrown off by rogue cosmic rays. One client of mine had to teach me a new word to measure the size of their data set, weighing in at a trillion gigabytes; double-checking even a tiny fraction of their calculations would be totally impractical.
We rely every day on incredibly complex systems like these, so gigantic that we cannot hope to check against “ground truth”. Indeed, in many cases, like weather prediction or medical diagnosis, there is no way even to define the “right” output. But the impossibility of the task should not deter us any more than it did the logarithm calculators. My zettabyte-scale client built clever consistency tests that sanity-checked summaries of their software’s actions, and designed the system to be robust in the face of occasional errors on single data points. I’ve helped others to adopt pair programming and code review by business people to improve the accuracy of their algorithms. We can’t just give up and hope the machine is right, or we’re back to reading entrails – instead, aim to trust but verify, and set your engineers on the hunt for every error reduction they can find.
This first appeared in my weekly Insanely Profitable Tech Newsletter which is received as part of the Squirrel Squadron every Monday, and was originally posted on 12th May 2025. To get my provocative thoughts and tips direct to your inbox first, sign up here: https://squirrelsquadron.com/
