Is Software Done?

The choice behind how the gains from AI will be spent.

More than twenty years ago, my team lead at IBM taught me to play the board game Go. At first I was quite confused, because unlike all the other board games I knew, there was no clear ending condition. The game doesn’t end when the score reaches a certain value, or certain pieces are in particular positions, or all of your pieces are taken. It ends with a strange kind of consensus between opponents when both players agree that the game is done. That happens when each player looks at the board and realises that to make another move would not improve their position, and may even make it worse.

In anything at all, perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away…

— Antoine de Saint-Exupéry

I used to think software was like this.

Once you’ve started building, you quickly realise that anything, even the simplest idea for a program, can contain multitudes and the more you create, the more ideas you have for new features, or slight tweaks and improvements. Still, I believed in the theoretical possibility of a wise engineer spotting the point at which to add anything more would make the whole worse, and pronouncing that the program is done.

Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

— Zawinski’s law outdated in specifics, and it turns out excessively optimistic

The Iron Triangle

The ‘Iron Triangle’ has been a project management staple for decades.

Usually, the idea is that for any particular level of quality, you can trade off between the constraints on the corners. So if you want to reduce cost, you’ll either have to increase time or reduce scope. Or, as I’ve seen it used on many software projects - the timeline has been reduced by diktat, so we need to reduce scope or increase cost (by adding more resource to the project - although see The Mythical Man Month for the limits to that approach).

Something that doesn’t happen often is to suddenly be told “a software feature now costs a fraction to deliver than it did before”. Now there are definitely arguments to be had about what that fraction is, and the costs of maintainance and where the new bottlenecks lie after widespread AI use, all of which I hope to pick up in a future post, but as Linus Torvalds says - it’s clear that it’s useful.

AI could wipe out half of all entry-level white-collar jobs — and spike unemployment to 10-20% in the next one to five years

— Dario Amodei, 2025

Apparently 99% of CEOs expect AI layoffs within two years, but this assumes a “lump of software” fallacy. The Iron Triangle tells us that there are a number of approaches to a sudden reduction in cost per feature:

1. We could keep budget the same and deliver the same projects at higher quality
2. We could deliver more projects for the same budget (time)
3. We could add more features and improvements to the same projects (scope)
4. The one that people seem to jump to - we deliver exactly the same amount of software at the same quality, in the same time, but reduce cost by firing people.

Now of course, the gains don’t all have to go to a single one of these options, in reality they’ll be spread out across them. What are the chances they’ll all flow to number 4?

A famous example of this not being the case comes from William Stanley Jevons in the late 19th century. He noticed that while you might expect that improved efficiency of steam engines in how they used coal would reduce the consumption of coal, what it actually did was increase it. As the efficiency increased, the number of places it was economic to use steam engines increased more, so overall demand for goal went up rather than down.

Elasticity of software demand

I have never managed a software development team and not felt that there were huge amounts of valuable scope or quality being left on the table. The constraint has never been ‘we only have this much scope to build’, it’s almost always been ‘we can only spend this much money’ or ‘we only have this much time for good market positioning’.

Companies that fall for the “lump of software” fallacy will fall behind their competitors almost immediately. They’ll miss out on the users, who will go to the higher quality, more featureful offerings. They’ll miss out on the internal efficiency gains, as their competitors build the long tail of efficiency improvement tools they always wanted but never previously had budget for.

Some companies have already faced something like this. Whatsapp is, at core, a very simple product that was arguably ‘done’ by 2015, with 57 engineers serving 900M users, but since then the engineering headcount looks to have increased by 20x. Most people need only a tiny core set of word processing features, but Microsoft have been shipping substantial updates to Word every year for decades. Those teams don’t shrink and shutdown because they’re ‘done’, they instead grow as they find new features to deliver and improvements that their users (or corporate stakeholders) want.

The wise software leader who announces “now we’re done”, may be possible in some circumstances, but those are rare in the commercial world. There are just too many reasons to keep going.

In many ways, commercial software isn’t like the game of go. As a game progresses, the overall impact of adding a new stone to the board becomes more clear, while in software a new feature adds more uncertainy and potential. In go, the board is static and bounded, you can’t suddenly open new markets and jump across to points that didn’t exist before.

In 2019, Lee Sedol, the Go master, announced his retirement because AI was now “an entity that cannot be defeated”, yet broader interest among players surged rather than collapsed, with many of them relying on that undefeatable entity to help them learn and hone their skills.