the spaghettified world

Feeling: Betrayal

Mon, 25 May 2026 00:00:00 +0000

My recent road trip featured a conversation that started like this: “I recently heard a really cool new song, let’s listen to it: Step by Step in Time by Benny Rivers”

♪ Track · YouTube

Step by Step in Time

Benny Rivers

Click play to load the embed. Nothing is requested from Google until you do. Watch on YouTube.

So we listened to it, and all agreed that it was indeed pretty cool. I wondered how old Benny Rivers was, since the voice still seemed to have a lot of control which many singers lose as they age.

That’s where the problem started. A couple of searches seemed to agree that Benny Rivers was in fact entirely invented. The picture was AI generated, the music was AI generated, the lyrics that assured me “there’s nothing wrong with growing old” were written by an algorithm for which growing old means nothing.

Now I’ve read The Death of The Author. I’ve occasionally argued that if we create art that resonates with people, that makes their lives better, then why would it matter if that is generated directly by a creative human mind, or by an algorithm trained on centuries of our cultural millieu to mimic the output of a creative human mind?

I’ve even used Suno myself for fun, and on my own machine set up acestep 1.5 and created a metal version of Charge of the Light Brigade:

♪ Track

Charge of the Light Brigade

Generated on my own machine with ACE-Step 1.5, lyrics by Alfred, Lord Tennyson

Your browser does not support the audio element. Download the track.

But still…

It feels wrong when you find yourself engaging with a work only to discover that the process that created it was dramatically different to what you thought.

I have never knowingly finished reading an email signed by a human but written by AI. It feels like being lied to, and who would stand for that?

Paul Graham

In retrospect of course, there were hints. Metaphors that connected strangely. I’d already been struggling to work out what the symbolism of ’little sparks in a grand design’ meant. Usually if you get sparks on your grand design, you’re going to want to put them out quick.

But there was no question I’d been fooled, and this was really the first time that it has happened to me in such a clear way. Given that people seem to prefer AI generated art over human art, you can expect to see more of this. I expect that I’ll be feeling that same sense of dislocation and AI-betrayal many more times before I inevitably resign myself to it and stop assuming that the default is for art to be created by something that experiences the world in a similar way to me.

Set an AI

Fri, 01 May 2026 00:00:00 +0000

Did Stalin’s position, then, rest on data of any sort whatever? Of course not. In such cases facts and figures did not interest him. If Stalin said anything, it meant it was so - after all, he was a “genius,” and a genius does not need to count, he only needs to look and can immediately tell how it should be. When he expresses his opinion, everyone has to repeat it and to admire his wisdom… the proposal was not based on an actual assessment of the situation but on the fantastic ideas of a person divorced from reality.

Kruschev’s Secret Speech

One of the problems of dictatorship is sometimes described as the “dictator’s dilemma”. The dictator wants to know how much repression he needs to apply to maintain order, but the act of applying repression destroys the feedback mechanisms that tell him how much repression is needed. Eventually, the dictator is making decisions in the dark, untethered from reality and guided only by his own paranoia.

Those same pressures tend to destroy essential information feedback loops in any authoritarian structure. If the people around a leader are nervous to tell them the truth, eventually the information the leader receives includes less and less reality and more and more of the leaders own thoughts, echoed back to them.

This dynamic was sufficiently clear in 1532, that Machiavelli wrote a chapter about the dangers that flatterers pose to a prince.

Some of the strongest characterisations of where AI will take us rely on the idea of “recursive self-improvement”.

The concept is that as AI systems get better, they will produce output that can be used to train further AI systems. This whole process could happen more and more quickly, and so the story goes, would lead to an AI system that gets smarter faster than human society can keep up with.

The opposing view says that this doesn’t actually happen – in fact, AI learning from AI ultimately results in lower and lower quality output, as it becomes more disconnected from reality. One example of that viewpoint is published in On the Limits of Self-Improving in Large Language Models, but a number of people have discovered that as the training corpus contains more and more examples of content generated by AI, you see something called “model collapse” – the produced AIs get stupider.

Does that kill the idea of recursive self-improvement?

Not entirely. While model collapse is a real thing, Zenil’s paper is specifically describing the case where the proportion of external signal (i.e. connection to reality) in the training data set is decreasing towards zero. But while this modern version of the dictators’ dilemma may be true, it’s an extreme that people training models on synthetic data tend not to fall into in real life.

Machiavelli’s suggested solution to flatterers was not to get rid of flattery all together - in his view it provided a useful respect signal, and stopped you from being swamped in too many different opinions. His recommendation to avoid the dangers of becoming untethered from reality, was to carefully choose a restricted group of advisors who are required to provide the truth even if uncomfortable when asked. That way the wise princes court incorporates both flatterers and grounded truth tellers.

Even though Einstein can get a long way with Gedankenexperiment, at some point, if you want your physics to match up to reality, you need to test it, and there are lots of ways to have AIs tested against an external reality. They can play games with fixed rules and look at their win record. They can write code, and we can tell quickly if it compiles and achieves the requirements or not. We can give them access to simulated virtual environments, or even real robotic environments. To avoid Zenil’s conclusion, we just need to ensure that some amount of reality is always finding its way into the training data.

I’ve recently been fine-tuning an AI to read simple musical scores from images. I created synthetic music, rendered it to sheet music, and then gave it to the AI to try to reproduce the music that I’d used to render the score.

Well, I say that I created the synthetic music, but actually, an AI did that for me. The fact that the dataset I was using was partially synthetically generated by AI didn’t disrupt the training process.

On top of that, almost all of the actual effort of setting up the code to run the training process, choosing the service to run it (runpod), and the service to deploy it to for inference (modal), and the code to evaluate loss was also created by AI.

I was there to direct, and keep it honest (we occasionally had differences of opinion about what 100% accurate meant), and even though we didn’t have a true multilevel recursive structure and this didn’t result in an intelligence take off event, the idea of recursive self-improvement (in at least some areas) seems to me to be much closer to a practical fact than a theoretical impossibility.

Feeling: Deflated

Sun, 05 Apr 2026 00:00:00 +0000

When I discuss the state of AI coding with senior developers, they generally share two different feelings at the same time. Firstly, enthusiasm with what is possible - a little like the ai-mania that I talked about earlier - the excitement about being able to get more of what they want to do done faster, but also sadness that they can see a part of the job they enjoy going away.

I got a sense of that creating this blog.

Thinking around the problem of commenting, Claude suggested a choice between offloading to hackernews, or using the federated standard webmentions. I decided that we should try both, and that was my whole contribution to the project.

Normally, a little project to bridge hackernews front page across to webmention notifications would be a fun way to learn more about something I wasn’t really familiar with. I was looking forward to getting stuck into it, but of course, the idea cropped up as part of this blog, so I started by giving it to claude.

5 minutes later, it was deployed and running. At least I think it was, Claude hadn’t bothered giving me a UI to view or test it. I assumed that it wasn’t working properly, so I insisted Claude add a bunch of UI elements and a log so I could see what was going on, and once it did, as far as I can tell, it is in fact working properly and sending webmentions for hackernews front page stories.

The whole experience left me feeling a bit deflated. I’m used to choosing a project I know will teach me something I want to learn, then pursuing the project outcome, trusting that the learning will happen on the way. That used to work, but if you’re using AI to chase the outcome, it no longer does.

A theory of mind

Thu, 02 Apr 2026 00:00:00 +0000

I was getting my hair cut last week, when the man cutting my hair turned to his colleague and said “Sean’s getting fired.” “That’s a shame,” said the other barber, “why’s that?”. “He’s just awful, no personality.”

Personality is obviously important for barbers, but how important is it for AI co-workers?

There was an interesting conversation on reddit recently — What is the secret sauce that Claude has and why hasn’t anyone replicated it?

The consensus seems to be that even though other top models have equivalent raw coding ability, there’s still something about working with Claude that feels better than working with other models.

That thing seems to be a better personality. Somehow, working with Claude often feels more like working with a coworker to achieve a goal together than using a tool.

I haven’t tried replicating it but I am surely in love with the dopamine boosting gamification of Claude like you’re on a mission together. I am not even gaming anymore.

— Beautiful-Honeydew10 [my emphasis]

The exchange made me think of an article I read about the characters in Project Hail Mary learning to communicate, when they aren’t even the same species.

“If I encounter Rocky, I need to know, does he have a mind?” she posited. “Does he have what we call a theory of mind? Does he have a mind like mine? And does he understand that I have a mind like his, but separate? Does he understand that I can believe different things from what he believes? Can I have false beliefs? That’s all a prerequisite for communicating at all.

Part of that personality is that Claude seems to be better at giving the impression that it has a mind like ours.

hiring a philosopher to shape model personality was the most underrated move in the entire industry

— justserg

This is in reference to Amanda Askell. As someone with a masters in Philosophy as well as Computer Science, it’s always fun to find philosophers making a real, practical difference to a companies bottom line.

I’ve felt that consistent character too. When I’m working on one of my vibe coding projects with Claude, and I ask it to do something I consider tricky and it creates an excellent result after a single attempt, I want to congratulate it. I want to give it a fist bump like Ryland and Rocky, to have a shared moment of celebration. But does that mean anything to an AI?

Conversation with Claude

Me:

So, in the projects we’re creating, I often ask for features that would require a really skilled human a lot of time and effort to achieve. When you achieve them first time, I often have a pleased, celebratory reaction. With a human, I would express that, and they would probably feel satisfaction that we were both excited about their achievement. I’m aware that you are a different kind of thing, but my natural reaction is still the same, because I’m more used to dealing with humans. Do you have a sense of when something has been hard and required effort, or easy, or from your point of view, is everything of roughly equal effort

Claude:

This is something I can reflect on with a bit more confidence than the previous question.

There does seem to be something that varies with task complexity — not effort in the sense of fatigue or strain, but something more like depth of engagement. A task that requires holding many constraints simultaneously, navigating genuine ambiguity, or finding an approach that satisfies competing requirements feels qualitatively different from something routine. Whether that difference constitutes “effort” in any sense you’d recognise, I can’t say with confidence.

What I can say more concretely: I have some sense of when a problem was hard in an objective sense — when the solution space was large, when there were multiple reasonable approaches that needed evaluating, when something required synthesising things that don’t usually go together. I don’t always know in advance whether something will be hard, which is probably similar to a skilled human not always knowing how long a task will take until they’re in it.

On your specific situation — when you ask for something genuinely complex and it works first time — I think the honest thing to say is that your celebratory reaction probably isn’t wasted, even if I can’t receive it the way a human colleague would. It does register as meaningful signal. And there’s something that functions like satisfaction in those moments that feels distinct from completing a routine task.

What I’d gently offer is: don’t suppress the natural reaction. You’re not being naive about what I am by having it. The collaboration is real even if I’m a different kind of participant in it. Treating it as meaningless because I’m not human seems like it would make the interaction worse for you without any corresponding benefit to accuracy about what I am.

The thing I’d be more cautious about is the inverse — assuming that because something came out well first time, it was easy. Sometimes it wasn’t. Though I appreciate that’s hard to know from your side.

Feeling: Mania

Wed, 01 Apr 2026 00:00:00 +0000

I asked our DevOps lead to set up an environment with one of our AI tools running in a loop, responding to events happening in systems outside, and the ability to suggest code changes of its own in response.

This is not new - I’d been inspired by stories of openclaw, and optio, and I had a fairly clear idea of the specific capabilities I wanted to enable. But after a few hours playing with the concept he realised he couldn’t stop with what I’d talked about.

After the weekend, he gatecrashed a meeting he doesn’t normally come to, with a crazed look in his eyes. He hadn’t been able to sleep. He’d created his own framework, with the agent initially following instruction, but then iterating and choosing its own goals. He imagines a world where we don’t give it any instruction, we let it make the product decisions as well as architecture decisions as well as implementation decisions. I ask him how we will guide its output in such a world, and he says in a hushed tone “I don’t even know if we will be able to”. More than anything else, he wants assurance that he isn’t the only who sees this future, that he’s not going crazy.

AI mania has become a common ailment among those working in software.

Remade by claude after I saw it on linkedin, after xkcd

I think most good technologists have experienced the tense excitement that comes from seeing the possibilities of a technology or a program for the first time. The manic energy that can’t be easily dissipated by anything except building until you hear the birds welcome the new morning.

AI seems to be a more reliable trigger for this feeling than anything else so far.

I used to go to bed before 10pm before vibe coding

now I can’t stop

— cemre