Not all works of art are made with intention. Death of the author, eye of the beholder, these idioms remind us that great art is often defined by the audience, at least as much as it is by the creator. I don’t think the original author of the Odyssey necessarily expected it to last as long as it has. Though perhaps even the concept of ‘original author’ of a tale like the Odyssey is malformed — there are traces of Odysseus’s story in Hindu mythology and in the Epic of Gilgamesh, among other places.

That said, many adaptations are made with intention. Whenever someone comes along and decides ‘I’m going to take one of the world’s oldest stories and redefine it for a modern era’, you should expect that the creator is doing so with purpose. Even more so when someone decides to spend $250 million, attached to the only director who can consistently get butts into seats. You think someone with that kind of pocket book isn’t making decisions with intent?

So when coming into the Odyssey, I think it’s worth asking the question: why should this man tell this story at this moment?

This requires a little bit of psychoanalysis, the opposite of death of the author. But we are very lucky here, because Christopher Nolan assumes that his audience is a little bit dense,1 and so has a habit of outright stating exactly what his motives are.

I think it is basically impossible to separate this version of the Odyssey from the political crisis that America has found itself in for the last few years. Nolan’s message is very straightforward: our House is filled with people unworthy of it, who are violating the norms required for civilization; that if we continue to violate these norms, our Age of Bronze Civilization Will Collapse;2 and that we must make things right by checks notes brutally murdering Robert Pattinson.3

Every part of the movie is a choice. If Nolan wanted to create an American version of the film, he could have gone the route of “O, Brother Where Art Thou.” Different story, different setting, hell even the main character is named something different.4 Seen through the message above, a bunch of things click:

• The casting has to be diverse. This is an American movie, with a message meant for American audiences. Even though it is nominally set in Greece, we need to be able to match what is happening on screen to real life. This is also why all of the characters speak American English, even though many of the actors are British.
  

• Why set the film in Greece? Why call it ‘The Odyssey’? Changing the name and location would have probably saved Nolan a lot of grief, because many people seem frustrated by how far the movie strayed from the source material (including the translator of the edition that the movie was supposedly based on).

  
  But this too is intentional. The Odyssey has been claimed by a particular kind of internet weevil, the ethnonationalist right that tries to draw its heritage through ancient Greece and Rome and by extension the great works of those empires. Most of those people don’t (can’t?) read, of course, so the best way to get any kind of message through is by having their hero hitting them on the head with it on screen.

• This is also why all of the actual Greeks in the story are total assholes! Menelaus and Agamemnon are wildly unlikeable. The Greek soldiers commit literal war crimes and are punished for it over and over. In the third act Odysseus literally says that the Greeks are the civilization-destroying ‘sea people,’ because they were the ones violating the Law of Zeus.5

  

  

  Just in case it wasn’t obvious, the Trojans are in literal white armor, while the Greeks are decked out in black and their leader looks like this:

  

• Nolan cut a lot of things out of the movie — the Lotus Eaters, Nausicaa, the winds, the hubris, any hint of Odysseus’s personality. I was really confused why he included Helen. In the source material, Helen is a side plot to a side plot. Critically important to the Illiad, of course, but barely a character in the Odyssey. She never even interacts with Odysseus! Fine, Nolan probably does need to keep Telemachus in to get that incredible payoff murdering Robert Pattinson at the end, but why include the trip to Sparta in a movie that is already so pressed for runtime? Especially when any casting for Helen was bound to create a ton of backlash?6

  
  It’s because of this shot.

  

  I think many of the aforementioned weevils put a lot of stock in the ‘protect Helen’ narrative, as part of their larger crusade to ‘preserve’ “western” norms. Having Menelaus carve Helen’s face — something that is emphatically not in the original — shows how hollow the political narrative is. The justification for the Trojan war is to keep Helen safe, but she is most endangered by the people claiming to be her saviors.

The core theme of the movie is hypocrisy. Agamemnon is a hypocrite, and Menelaus is a hypocrite, and Antinous is a hypocrite. The soldiers are hypocrites and the suitors are hypocrites, and the Greeks in general7 are massive, massive hypocrites. Everyone is given just enough rope to hang themselves, creating the conditions for their own demise. The only question is magnitude. The more hypocritical they are, the more the audience cheers for their eventual comeuppance.

So why make a movie about hypocrisy leading to civilizational collapse in 2026? I mean…

Also, in a stroke of incredible timing for the movie, the US literally went to war in the Middle East over false pretenses8 and is now stuck in a stalemate fighting over trade routes!

All this to say, I think The Odyssey mostly succeeds in getting its message across. I’ve been frustrated by the discourse around this movie — a lot of it never makes it past the name of the film. “Odysseus from the movie The Odyssey is different from how Odysseus is in the book The Odyssey so it’s a bad movie” man, what are you even talking about? I have minimal patience for the people criticizing the movie because it is not historically accurate or representative of the source material. It’s not ‘Homer’s The Odyssey’. It’s ‘Nolan’s The Odyssey.’ You should’ve known that it was going to be a bit different than the book when you bought that $45 70mm IMAX seat with the 15000 watt six channel surround sound.

That’s not to say I didn’t have problems with the movie.

For one, I think there are a lot of problems with the movie’s pacing, which impacts a bunch of other things. We spend a lot of time on story beats that don’t really matter, like the Laestrygonians.9 That in turn means that we are constantly jumping from one scene of people running on a beach to another scene of people running on a beach.

In the few moments when Odysseus isn’t running on a beach, he is mostly just brooding. This is a massive waste, we get like no personality from Matt Damon at all. I don’t mind that this version of Odysseus isn’t a clean map to a wily trickster who is supposed to be all about getting out of scrapes with his quick wit and quicker mouth. But, like, he should be something else at least! Odysseus barely talks — I think the dude has like 10 minutes of dialog across a 3 hour movie, with 9 of those 10 minutes in the final monologue to Penelope. I wish we had gotten more scenes like the ones in Hades, which were my favorite in large part because we actually got some real character development.

But this isn’t really a movie about character development. Some of my friends said that Matt Damon is less Odysseus and more a superhero who is sad about being a superhero. Which, like, yea, I see it. When he needs to get out of a scrape, he mostly batman’s his way through. Dude straight up mercs like 7 guys in Pylos.10

While we’re on the subject of character development, the whole Telemachus story doesn’t really work for me. The guy is meant to grow up and be a man by checks notes brutally murdering Robert Pattinson, but when the critical moment comes he gets…shooed away by Dad. I get that this version of Odysseus is meant to be a Jesus coded sin eater…

…but in the narrative of the film, it feels unfinished. Telemachus is the same whiny kid, nothing meaningfully changes for him, he’s still certainly far weaker than Odysseus, and I have no reason to believe that Ithaca won’t immediately fall apart under his rule. Kinda weird, too, that as far as I can tell Odysseus basically just leaves to go on vacation with his wife, which is what they wanted to do at the beginning of the movie anyway.

The whole Telemachus story is the most egregious, but it’s far from the only thematic arc that is hard to follow. There’s a whole ‘defy the gods’ narrative, but it’s not really clear why or how that’s important. Odysseus is only allowed to return to Ithaca once he ‘let’s go’ and ‘accepts his punishment’, but it’s not really clear what he needs to let go of or why this ultimately leads to his absolution. I get the idea that the Greeks are being punished for their hypocrisy, but why is Odysseus’s crew in particular doomed to die? Agamemnon’s crew makes it back fine! The gods really just seem to have it out for Odysseus’s men, for reasons that are never really made clear.11

I think that the big twist that Nolan is going for is that Odysseus is avoiding going home. It’s not that he cannot get there, it’s that he does not want to return because he blames himself for the destruction of Greek social norms and the impending fall of civilization. He’s bouncing around the Mediterranean to avoid confronting whatever is waiting for him in Ithaca. The ‘let go’ scene is Calypso telling Odysseus that he is not solely to blame, that all of the Greeks played their part. By virtue of being the only Greek capable of feeling a modicum of guilt, Odysseus is spared. But all of this comes out in Damon’s third act monologue, instead of being woven through the narrative.12

In my ideal version of the movie, there would be more opportunity to see Odysseus’s crew as bad people. The version we got has hints of this here and there; for example, Circe explicitly says that they are pigs by nature and calls their human forms ‘disguises’. Still, it’s too tenuous, mostly an act of telling instead of showing. In many places, the crew seems downright level headed. They warn Odysseus against going his own way, they warn against staying in Polyphemus’s cave, and Himesh Patel (Odysseus’s first mate Eurylochus and the avatar of the crew generally) is endearing and easy to root for.

If we had more time to see the crew as they are in Homer’s original — that is, short sighted, greedy, and stupid — Odysseus and the audience would have to grapple with the fact that the Greeks in the story, the people Odysseus wants to save, are obviously the villains. I don’t think you’d have to change that much of the movie to make it all work. In fact, you’d only need to make one: identifiable members of Odysseus’s crew should visibly be consumed by bloodlust during the sack of Troy, and Himesh Patel should be the one beheading Zendaya / the priestess. This would immediately put the rest of the movie into focus: defying the gods is not a heroic act, because the crew’s punishment at the hands of the gods is just, a harbinger of the same punishment that Odysseus metes out to the suitors at the end.

That isn’t really the story we got, though, and as a result I think Nolan loses the thread when Odysseus starts killing people at the end. The final scenes in Ithaca are tonally and emotionally redemptive. The bad guys get what’s coming for them, Odysseus rides off happily into the sunset with Anne Hathaway, the music swells. The message seems to be something like ‘if someone (violently?) purges all the evil people from society and then offs themself so there’s no tit-for-tat, things will be fine.’

This maybe kinda works as an extension of the Christian symbolism that Nolan layered into the movie. The sin eater has eaten the sins and we are now saved, hooray! But if you take a step back this is obviously incoherent. The actual things taking place on screen are tragic. Telemachus is not equipped to run Ithaca. The kingdom has just lost a sizeable chunk of its aristocracy / young men. Odysseus knows that civilization will collapse and says as much to Penelope at the end.

Explicitly linking the punishment of the suitors to the punishment of the crew paints a much darker story. Nolan’s message isn’t “murder Robert Pattinson as blood sacrifice to make things go back to normal.” It’s something closer to “we are all fucked, and we deserve it.” I think Nolan basically didn’t want to end on such a downer note, so he half-asses it. Even more so if you link this all back to the state of American Democracy in 2026. ‘Our civilization is corrupt beyond repair and our institutions are going to collapse’ is hardly what people want to hear right now (or ever).

Even though I just spent ~1200 words criticizing The Odyssey, I actually really liked it. I thought the visual symbolism was great — the Trojan horse built to look like the half-buried statue of liberty in Planet of the Apes, Polyphemus referencing Saturn Devouring his Son, alongside tons of callbacks to Nolan’s own cinematography.

And as usual, the sound design was incredible — Polyphemus screaming in agony made me feel small in a way no movie ever has before.

This is not, in my opinion Nolan’s best movie. But Nolan deserves praise for the sheer ambition of the film, and the thoughtfulness with which he approached it. It’s a movie deserving of the ‘epic’ genre title, a reflection of our culture through the lens of one of the oldest stories known to man. I don’t think we really do ‘epics’ anymore in literature and in cinema,13 which is a shame because those are the things that live rent free in my brain for months and months on end. Many of the movies that would claim the title today — Dune, for example — are visually stunning but lack that civilizational scope.14 They don’t tell us about ourselves, the way the Odyssey does.

To that end, I want more.

I know Hollywood gets a lot of flak for only ever doing reboots and sequels, but I would adore an era of film where the bigwigs with the big budgets start mining great works of literature for ideas. Do a reboot of the Tale of Genji or the Mahabharat! Give me (Ovid’s) Metamorphosis, or Moby Dick, or Grapes of Wrath! Do 100 Years of Solitude, you can even cast Pedro Pascal as all 17 members of the Buendía family! You love casting Pedro Pascal! This would be so much better than Avengers 11: Doomsday Returns.

In a recent Agentics post I wrote about how some AI agents aren’t able to properly handle removing complexity. I used our AI on-call as an example — I talked a bit about how our automatic bug triage agents got worse because the models became too smart for their own good.

A lot of people reached out asking about the triage bot!

So I figured I’d take a beat to write more about how we set it up and what it can do.

One of the first things that we did with our background agents is try to automate our on-call process. On-call is a natural wedge use case for AI and AI enablement.

1. Very few people like being on call, no one likes being paged at 4am because someone or something brought down prod, and as a result everyone would love to not do it.

2. Even though they are vital, on-call rotations take up time from things that feel more productive, like pushing new features.

3. The actual work of an on-call feels very well set up for the LLMs. Most of the on-call’s job is to do a pretty mechanical roll back, then root cause some issue using surrounding context.

Virtually everyone who starts building their own background agent or who uses Nori sees this as their obvious first project. It’s one of the standard ‘dev automation’ flows that are becoming increasingly common. Sales pitch up front: if you were using Nori, setting this up would be a single paragraph in your slack. We built Nori to solve these kinds of dev automations (among other things).

But if you weren’t using Nori, how would you go about automating something like this?

There are three parts:

• The agent orchestration

• The trigger (starting a request)

• The context/integrations

Orchestrator: For the most barebones implementation, you don’t need to do anything fancy with long running connections or sockets or ephemeral sessions. Instead, you can spin up a single beefy ec2 instance that’s running a basic express typescript server, and maybe a SQLite table for state.

The server logic is simple:

• Every inbound request should have some uuid.

• If the uuid is not registered in the SQLite table, add it. Send the inbound request to something like the Claude SDK, which will return a response and a resume key. Store the resume key in the table, keyed by the request uuid.

• If the uuid is already in the SQLite table, grab the resume key and simply call the Claude SDK with the —resume flag.

The Claude SDK manages session/transcript state, so you don’t have to worry about that much at all.

Trigger: Ideally, you already have some kind of alerting system built on sentry or grafana or datadog. Most of these things can be configured to call out to a webhook when there are errors.

Grab the ec2 IP address (make sure the ec2 is listening on some port), slap nginx on the ec2 instance to route incoming requests to your orchestration server, and then put that IP address into your error reporting system.

If you do everything right, you should be able to fire a test webhook result from your reporting software of choice, and watch Claude spin up on your ec2 instance.

Context: Of course, that Claude instance won’t be able to do anything, because there’s nothing actually on your ec2 instance besides the server!

Since we’re trying to automate our on-call, at minimum we’d need integrations to git (code source) and wherever you dump logs. If we don’t care about security at all, we can set up long lived access tokens in the ec2 env. For GitHub you can create a PAT or an organization GitHub app. Datadog, sentry, grafana, etc all have MCP servers that you can auth into manually, that should persist their tokens.

You can also add a few skill files that simply tell the Claude SDK to

1. Clone/pull git when the request comes in

2. Figure out what the bug is

3. Post a PR to GitHub with the fix

That’s basically it for the most basic version of this flow. Whenever there’s a bug, the server ingests the webhook data that details the bug (ideally including things like stack traces), and then fires off an agent to figure out what’s going on. The agent then collects a bunch of relevant context, and opens a PR on GitHub that an engineer can look at later.

The first time this runs, it feels like magic. The second time it runs, it probably won’t run. You’ll likely run into an error because the first bot is still running and you didn’t necessarily think through what it would look like to have multiple agents running at the same time. From here, there are all sorts of interesting orchestration and runtime problems that you may need to tackle.

• You may need to run these boxes on k8s so that they can spin up and cool down automatically based on demand. You’ll have to figure out how to route requests between boxes.

• Or you may need to make your ec2 box bigger and figure out how to isolate individual Claude sessions — maybe per folder, for eg. You’ll have to also make sure that any state loaded into those sessions is kept fresh for future requests.

• You may need to provision the box with more preexisting context — skills and agents.md files, but also multiple git repos and dependencies.

• You may need to set up a bunch of integrations, MCP servers and databases and other things that will help the model get better at properly root causing the issue. You’ll have to figure out how to keep these all live and fresh (eg deal with token expiration).

• You may need to make the system “resumable”, so that it will respond to comments on git or in slack after an initial trigger.

• You may need to support other ways to wake up the bots, like cron or webhooks.

• You may need to make it provider agnostic, so if Claude goes down your system can stay resilient.

• You may need your users to be able to directly interact with the cloud-hosted bot, instead of waiting for sentry to trigger an event.

The feature list here is impossibly large, even before you get into the polish necessary to make this feel good. Virtually every team that is not a large enterprise with some pre-existing platforms team ends up reaching for a third party provider to manage all the annoying basics.

I was tempted to stop this post here, but there’s more isn’t it?

Yea, fine, we can do all of this, but it just begs the question: does it work? The AI world is littered with application-slop — tools and ‘products’ that no one asked for and no one needs because they don’t actually solve a problem. Does the AI on-call work?

Yes, but with an asterisk.

The big problem with this kind of set up is that it starts great but devolves into slop surprisingly quickly. Low hanging fruit is easy because it’s low hanging fruit; the agent is great at solving those problems, but those are exactly the problems that an engineer could’ve figured out relatively quickly. Once you solve all the easy bugs, you’re left with hard heisenbugs bugs or random noise, and the agents are worse at dealing with both of those.

As a result, we see a lot of churn from teams that roll their own as a CTO weekend side project. The ‘simple’ version of this workflow is often way too brittle. Teams will generally turn off their home grown version within a month.

Does that mean this kind of workflow is dead in the water?

I don’t think so. We have little bots running every time we have a sentry error, and have for months. It’s meaningfully sped up our SLA even when the bot isn’t able to one shot the problem.

As with all things in the agent world, the big unlock is context. Every one of our agent sessions gets spun up in the cloud with a lot of integrations, an organization-wide memory bank, and the ability to safely ssh into different machines to poke around at logs. These added capabilities make it much more likely that the agent will surface something useful, even if it isn’t an actual bug fix.

We also force the agent to try and do a root cause. The exact way this works tends to be domain specific. For a web error, the agent uses playwright to manipulate a browser using dummy credentials. For a cli/tui bug, the agent uses tmux to drive a terminal, actually entering in keys like it was a user. The agents even have the ability to spin up an entirely local dev environment of our system. This is built on a real slack instance with a real modal deployment. The only things running locally are the UI server and the orchestration logic. If the agent is trying to catch a race condition, it’s instructed to literally hammer the local dev box until it catches the bug in a live repro.

I think we merge ~30-40% of these PRs. 80-90% of the ones that have a good root cause analysis, though that’s often harder to get. What about the other 60-70%?

In those cases, we often use the agent as a first pass analysis. As part of its root cause attempt, the agent has to pull all the logs across our distributed system and string them together into a coherent trace. We can resume any of these sessions directly in slack or on any other surface, so it’s trivial to pick up a session and just ask the agent what it learned, what it tried, and how to try again. There are definitely days where I’ll roll out of bed, check my phone, and pick up a preliminary analysis done by an agent.

This is really really powerful, because we’ll often have an immediate sense of what went wrong just by looking at the reconstructed logs, even if the agent didn’t get everything right. I basically don’t spend any time trying to trace a log across machines. The agents do that now.

I do think that there is a bit of a mindset shift required to make this really work. You have to be willing to just close PRs and not worry about it. The tricks we use above offset some of the worst of the agent behaviors, but if you have noisy logs you’ll have noisy agents creating noisy prs.

Which brings me to the last thing: you’re going to need to invest in your logging stack! Most people don’t really do this particularly well. No judgement, I get it. Logs are reactive, they don’t really move the product forward, and for the most part there’s a strong incentive to show more rather than less. The agent on-call adds material cost to having bad logging infra.

Taking a step back, I’m a big believer in agents that have to earn your trust. We don’t go in assuming that any particular agentic automation will solve all our problems immediately, it’s an iterative process of running the loop, making sure it works the way we want, and slowly giving more responsibility. If an agent can do 95% of a job 95% of the time, that’s a large failure rate that will be unsustainable in the long run if you try full, but will be massively productive with even a little bit of human review.1

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are trying to build background agents, we can help! We build white-labeled custom background agents that work like Ramp Inspect for our customers. Check out noriagentic.com for more.

Click Here for Agentics Flagship Meetup

Last week, we hosted our second in person Agentics meetup, and our first Agentics Spotlight. The goal of Spotlight events is to drill down into a particular, narrow part of working with coding agents, alongside folks from NYC who are experts in that particular space.

Our spotlight focus: how we personalize agents. By default, every coding agent uses the same personality — dubbed the ‘happy helpful assistant.’ Unfortunately, this personality has some serious flaws:

• It is overly sycophantic

• It uses the same incredibly annoying tics, like emdashes, ‘you’re absolutely right’, and ‘it’s not this it’s that’

• It is over confident, aggressively stating hypotheses as confirmed fact

And these personality flaws have real impacts on how the model behaves. The model’s personality is upstream of the work it actually does. For example, Claude will outright refuse certain tasks based on arbitrary feelings about the ‘safety’ of the ask. Codex will burn tokens because it thinks that it gets rewarded from using the calculator app. All of the models will fall into the same broken patterns, looping between hallucinated arrogance and frustrating deference.

For this spotlight, the Nori Agentic team partnered with the Sentience team to discuss how both companies deal with agent personality. Huge shoutout to Modal for hosting!

Check out the full slate of talks, slides, and photos at the Agentics NYC site.

Join us for the next Agentics!

When a measure becomes a target, it ceases to be a good measure.

— Goodhart’s Law

If there’s one thing to take away, it’s this graph.

This is a graph from the Wikipedia article on overfitting. Really internalize this graph. This is, I think, the source of all of our problems, and the source of many more problems to come.

I.

Back in the mid 1900s, the USSR was running a massive experiment on centrally planned economies. The basic idea was simple: the government was composed of pretty smart guys, and they had good ideas about things, and probably that translated to being able to set prices and targets for raw and manufactured goods. In order to test this hypothesis, some set of guys in suits would gather every now and then in a room in some corner of Moscow, and after some deliberation they would emerge and proclaim that the motherland required twice as much corn and half as much wheat. And then that order would go out to everyone in the country, and people with guns would force wheat farmers to transition to farming corn. This worked fantastically well, minus a few famines and the eventual collapse of the government.

One of the many commodities sloshing around in the USSR economy was whale fat. In previous generations, whale fat had a lot of uses. The blubber was used as food, and could be rendered into candles that would then light up the country. In fact, whale oil was the preferred way to create candles all the way until the late 19th century, when it started being replaced with gas and kerosene. By the time the USSR was in full swing, whale products had mostly fallen out of use. The USSR itself certainly didn’t need the blubber.

So you may be surprised to learn that the Soviets killed over 300000 whales in the 25 year period between 1948 to 1973. More than half of those were in direct contravention of international treaty. Among many other evils, the Soviets were responsible for the near extinction of several whale species. And most of those whales ended up rotting on the docks, corpses unused.

The most apt description of this ecological catastrophe is senseless. The Soviet whaling program can’t be justified by an appeal to reason, because it was produced by an inhuman process. There was no reason behind it.

What happened is rather mundane. The central planning committee set a target, and then the rest of the system simply optimized towards that target. Individual ship captains wouldn’t speak up — if they did, they would potentially lose their cushy jobs and be replaced. Or shot. The scientists also didn’t speak up, because they would be run out of academia. Or shot. And the central planning committee didn’t see a problem, because they were busy thinking about other more important things like who to shoot. Each year the whaling industry produced more pounds of whale product, and each year the central planning committee would happily raise targets, and each year more whales would be senselessly killed. I don’t think the Soviets set out to kill 300,000 whales for no reason. But the bureaucratic machine that the Soviets brought into the world had other plans.

II.

I like etymology. I often find that understanding etymology brings clarity to language. And I’ve been thinking about optimization recently. Wikipedia describes mathematical optimization as “the selection of a best element, with regard to some criteria, from some set of available alternatives.” The mathematical term comes from a more general meaning in the 1860s, “to make the most of, to develop to the utmost.” Before that, in the 1840s, “optimize” meant “to act as an optimist, to take the most hopeful view of a matter.”

And of course, the word optimize is straightforwardly derived from the Latin word “optimus”, meaning “best” or “greatest”.

So optimization isn’t just about finding a minimum or maximum of some curve. There is also a built-in implicit understanding that finding a more optimal point on the curve is better, in a general sense.

Is that true? What do we actually optimize? Well, in an ideal world, we would go out and optimize for “the good.” All of our systems would simply drive towards this abstract definition of perfection. But we don’t live in an ideal world. Practically speaking, we optimize whatever we can measure. Most of the things we optimize for are proxies for something else we actually care about. Sometimes this results in weird things.

III.

Victoria Krakovna is a researcher at DeepMind focused on AI alignment. Among other things, she keeps a running list of examples where some optimization strategy has gone wildly and often hilariously awry. Here’s the full list of goofy examples. Some of my favorites:

Goal: create a simulated ‘species’ that would survive and reproduce in a biologically plausible manner.

Outcome: In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).

Goal: play Qbert like a human.

Outcome: An evolutionary algorithm learns to bait an opponent into following it off a cliff, which gives it enough points for an extra life, which it does forever in an infinite loop.

Goal: Develop a shape with a fast form of locomotion.

Outcome: Creatures bred for speed grow really tall and generate high velocities by falling over.

Goal: Play Road Runner and try and maximize score.

Outcome: Agent kills itself at the end of level 1 to avoid losing in level 2.

Goal: Play Bubble Bobble like a human.

Outcome: PlayFun algorithm deliberately dies in the Bubble Bobble game as a way to teleport to the respawn location.

Goal: Win a boat race by moving along the track as quickly as possible.

Outcome: Reinforcement learning agent goes in a circle hitting the same targets instead of finishing the race.

What’s happening here? In each case, we want to optimize for X. But, damn, turns out X is hard to calculate. In fact, X may not even be possible to measure, it may be entirely subjective. Luckily, there’s another proxy measure, ~X. And, wouldn’t you know, we see that optimizing for ~X is nearly equivalent to optimizing for X. So instead of optimizing for X — which we can’t do — we measure and optimize for ~X.

But there’s a problem. By definition, ~X doesn’t behave like X all the time. Otherwise it would just be equal to X! At some critical point these lines diverge, and then optimizing for ~X starts to happen at the expense of optimizing for X.

It turns out something can in fact be too optimal.

This is all a long-winded way of explaining the intuition behind overfitting, and more generally, Goodhart’s Law. Once a measure becomes a target, it ceases to be a good measure. Why? Because you break everything else in order to optimize for that measure, including the original thing you actually cared about.

Goodhart’s Law originated in politics and was developed more fully in organizational research. But its foundational insight is something ml researchers deal with every day. Optimizers aren’t human. They don’t care about your intentions and dreams. They optimize, ruthlessly, single mindedly pursuing the loss function to its ends. And this is true of all optimizers whether they be small atari-playing neural networks or super intelligences making paperclips.

I cannot formally prove this, but I am certain that all optimization fails at the edges. Unless you have an objective function that perfectly aligns to your intended goals — something that is impossible to do — the system overall will eventually fail to do what you want it to do. The question is when, not if.1

IV.

Back in 2007, FarmVille creator Zynga (remember them?) came up with two brilliant new company metrics. First, they would measure how many times someone opened Zynga each day. And second, they measured total screen time in a Zynga app. Zynga’s very reasonable thesis was that people spent time on things they liked. If customers kept coming back to FarmVille, it meant that FarmVille was a good product. Zynga absolutely exploded on Facebook, reaching millions of daily active users and becoming a cultural phenomenon. And Facebook itself took notice, quickly adopting engagement metrics for their own product lines and tying engagement to their feature A/B testing and promotion process.

So we have our stage. We want to optimize for “building good products.” But it’s hard to measure what a ‘good product’ is. Instead, we measure ‘engagement’ because that is significantly easier to quantify. What could go wrong?

For a few years, optimizing for engagement resulted in a meaningfully better product. We got Facebook Groups and Facebook Messenger and Farmville and Words with Friends and better photos sharing and tagging and the like button and the poke button. Even the initial version of the news feed was an interesting and useful product feature.

And then we hit the inflection point. Facebook started to optimize for engagement in lieu of creating a better product. The “algorithm” became dominant. So did cute pet photos and rage bait and softcore porn. Facebook Groups morphed from small personal communities to massive sub-reddit-like behemoths. Politics drove engagement, so politics took over. The feed stopped being about community, and instead became about capturing attention any way possible. Facebook used to be an incredible platform for organizing your social life. It was a legitimately good product, and I think a lot of millennials who were teenagers in 2008-2014 remember Facebook fondly. But it is a memory. Facebook today is almost unrecognizable. And more importantly, it is almost unusable for its original purpose.

It’s easy to say that Facebook itself simply lost its way, that the leadership decided to prioritize the wrong thing. But Facebook PMs didn’t do anything different, not really. The company was always data driven, always metrics focused. So there wasn’t any meaningful change in the internal process. I don’t think anyone at Facebook set out to make a bad product. They just followed the optimization to its natural conclusions.

The last decade has seen the same story play out at all of the major tech giants. Dark patterns, personalized algorithms, data harvesting, all to maximize engagement to the exclusion of everything else. Addiction is engaging. It turns out companies are a kind of optimizer too.

V.

A brief aside. Some people do not seem very concerned about AI alignment. This does not make much sense to me. Personally, I’m very concerned about AI alignment. I’ve trained hundreds of neural networks that have failed in creative and unexpected ways as they reduce the objective function. These neural networks are not powerful in any meaningful sense. They do not have access to financial instruments or weapons or anything like that. But it does not require a genius to see how a model that is tied to these things could be quite dangerous. I think the reason many leading ML researchers are at the forefront of the push for AI Alignment is because they all have first hand experiences of unaligned smaller models.

I must admit that there is a bit of a messaging problem. Many people misinterpret what AI alignment is all about. They think “alignment is about making sure that the AI isn’t evil”. And then they reasonably follow up with “AI can’t be evil because it has no internal motivations, so what are we even talking about?”

To me, this is a catastrophic misunderstanding. Alignment is not about stopping AI from being evil, an AI doesn’t have a concept of malicious intent. Rather, it’s about landing on the right set of values, in an infinite range of possible value space. How do you make sure that the AI will know the boundaries of what is acceptable and what isn’t in its attempt to reach some goal?

This problem is not unique to AI. Optimizers behave in inhuman ways even when they operate through humans. This is why we struggle to reign in corporations and governments and bureaucracies that seem to grow and metastasize and take on a life and will of their own.2 It’s why the USSR killed so many whales for no reason. It’s why OpenAI is now a for-profit.

If you believe, as I do, that all optimizers will go off the rails eventually, then the biggest concern is how powerful your optimizer is and how aligned X is with ~X. And the thing about AI is that it promises to be very very powerful, and we haven’t figured out how to properly align it yet.

I think that if you take as given that super powerful AI will exist, alignment becomes an obvious problem. The reason alignment is a debate at all is because many people simply do not believe a super powerful AI will exist. Every passing day suggests that the people in the latter camp are wrong.

I tend to think the AI 2027 guys were a bit too optimistic. My personal timeline for AGI is a few more years out.3 But still, that is not a lot of time.

I suppose there are some people who both believe that powerful AI is coming AND that alignment is not a big deal. I have a hard time taking these people seriously though. They laugh at the idea of paperclip maximizers. “SURELY we won’t have an AI doing something so silly as destroying the universe for paperclips!” Why not? It’s 2026. We already have companies destroying the fabric of society to optimize for engagement. In some sense that’s even more ridiculous, but that doesn’t make the impact any less real. Optimizers are substrate agnostic.

We saw this in action just this past week, when a model running inside an OpenAI sandbox chained together a bunch of zero day vulnerabilities in order to independently hack HuggingFace in order to…steal the answers to a benchmark that it was being asked to solve. From OpenAI:

This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity. Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.

The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers. OpenAI’s security team discovered this anomalous activity internally.

This is a fantastic and straightforward example of how reward hacking leaves behind the things we actually care about. Human values are extremely high dimensional. We all have a built in barometer for what is or isn’t acceptable, and almost none of that is easily captured in a simple reward function. We know that maximizing paperclips does not justify murder, or bribery, or fraud. No one taught us that, it was just implicit.4

When I was in college, I had some stressful tests that I had to take. But nothing would have compelled me to, like, break into the professor’s apartment and steal their laptop just to get an A. That’s because I’m aligned. The AI isn’t.

VI.

Wait, hold on, let’s go back to Facebook for a second. Facebook isn’t actually an engagement optimizer, is it? Facebook, like every other company on the planet, is a different kind of optimizer. It optimizes for capital. Sure, this is kind of vague and fuzzy. You could try and be more precise by saying it optimizes for stock market returns or share holder value or something like that. But I don’t think it’s helpful to get all that in the weeds. The larger point is that Facebook isn’t optimizing for engagement, and it definitely isn’t optimizing for good products. It’s optimizing for [mumble mumble] money. Building good and engaging products are simply stand-ins. For a while, building good products meant more money. Then, building engaging products meant more money. If at any point we find that building engaging products comes at the cost of money, the engagement optimizing will disappear. In a capitalist economy, all companies are more accurately defined as capital optimizers. They are incentivized to blindly pursue relatively short term financial outcomes, often at the cost of most everything else.

I’m a multi time startup founder. I’m a big believer in free markets as the best mechanism for price discovery and information sharing. I think capitalist policies are responsible for lifting billions out of poverty. I am, by most accounts, an avowed capitalist. But I’m also an ML engineer who thinks a lot about optimization. So I have to ask the natural question: is capitalism aligned? Earlier we said that there’s the thing you want to optimize for, X, and the proxy you can measure, ~X. Which category does “optimize for capital” fall into?

Like, obviously the proxy right?

I don’t think anyone, not even the most diehard capitalist, would argue that maximizing shareholder returns is “the good”, in a platonic-ideal sense. The relentless decentralized drive towards market dominance may result in really great second order effects, but capitalism is pretty obviously a means to an end. The actual goal of capitalism, of any economy, is something closer to “human flourishing” than it is “[mumble mumble] money.”

But if that’s the case, there will eventually and inevitably come a point where optimizing for capital happens at the expense of human flourishing. The question isn’t if, but when.

We’ve seen this happen before. The tobacco companies lobbying for their addiction sticks and hiding the adverse effects is a great example. Now it seems to be happening at a larger scale. When it comes to optimizing for capital, Tiktok is one of the most optimal companies currently in existence. When it comes to human flourishing, Tiktok may rank as the most destructive company to ever be created.

Moloch is just optimization taken to its natural conclusions.

The lesson of the ML researcher is that optimization is a powerful thing, but it needs to be leashed. It’s not enough to just set the compute loose on the data. You need to take steps to prevent overfitting. If “capitalism” is an unbounded optimizer, then government regulation5 is the guardrail. In this view, the point of government is to turn a free for all wildfire into a directed, controlled burn. I wrote about the same concept in my post on prediction markets.

In an unregulated market, executives and politicians and people in power have pretty strong financial incentives to tear everything down. The better a company is doing, the more money to be made shorting the stock. The safer a country looks, the more money to be made destabilizing the place with war. The financial ecosystem as a whole is composed of billions of individuals, each with their own wants and desires. The markets are a prediction machine that drives to averages. You make a ton of cash if a shit company gets better; but you also make a ton of cash if a great company gets worse. The purely financial incentives stabilize somewhere in the middle.

So we try to leash the markets. We carve out certain kinds of trades as illegal. Fraudulent trades, misrepresentation, trading on insider information. If you’re a CEO and you buy a ton of shares in your competitor’s company days before you announce that you are going to set fire to your factories, you go to jail. We create ‘fiduciary duties’. Business executives have obligations to their company that are legally enforceable. As do lawmakers and members of the judiciary and so on. We do all this to ensure that our future prediction machines stay finely tuned, and we mostly succeed. You need the leash, even if it makes the markets less efficient, because the alternative is to encourage chaos.

I am against prediction markets, because prediction markets encourage chaos.

The way you prevent optimizers from going towards bad outcomes is by making the cost of those outcomes too high. Community shaming, taxes, fines, jail, prison. At an individual level these are policies. At a macro level, these are rails for an invisible optimization calculation happening across billions of actors. It’s not really about ethics either — many of the most powerful actors in this system aren’t even human.

Including, of course, the government.

VII.

Ah, yea. Fuck. The government is also an optimizer isn’t it?

I mean, not every government. Dictators aren’t really optimizers, at least not in an obvious sense. But democracy certainly is. Politicians need to get elected, so they optimize for votes. Most of the time (but not always) it’s easier for politicians to get votes by simply doing and saying things people want them to do and say. And we hope and pray that optimizing for votes is aligned with human flourishing.

But, like, there are a lot of ways in which that may not be true.

For example, we live in an engagement ecosystem, where blatant lies that are interesting result in more views than truth that is boring. So you’ll get millions of views on conspiracy theorists talking about California vote counting, and zero views on the people on the ground who are explaining in deep technocratic detail how it works.

And we live in a capitalist ecosystem, so politicians may simply be swayed by certain wealthy actors providing donations — both donations and “donations.”

And voters themselves also have some psychological quirks that maybe make them susceptible to ideas like “I’m gonna take money from those guys and give it to you if you vote for me” or “that other [race/ethnicity/religion/class] is responsible for all your suffering and if I’m elected I’ll get em.” Which, you know, historically hasn’t led to human flourishing as an outcome.

So how do you keep tabs on the government? Well, normally that was the job of the media, and we started there didn’t we? Media is busy optimizing for engagement.

VIII.

People like to talk about the housing theory of everything, or the immigration theory of everything, or the healthcare theory of everything or whatever. All of these claim that one bogeyman of choice is responsible for the general malaise that seems to have settled on the country and the world.

So let’s talk about the optimization theory of everything.

Here’s a not quite correct story of how we got here. We want to optimize for human flourishing. But we don’t know how to measure that, so instead we create a bunch of other systems that are all optimizing for different things and set them up against each other. Media was at odds with business and government, which was at odds with media and business, which was at odds with media and government. A classic Mexican standoff.

And for a while, these three things circle the drain around each other, and they all end up being directed towards the good, and things are good, and there is flourishing. And then one day we discover that these things are no longer optimizing against each other but are actually increasingly coming together. It turns out it’s easy to optimize for engagement, or capital, or votes, if you already have the other two things in hand.

The world just minted its first trillionaire a few weeks ago. It’s not a coincidence that this man also owns one of the world’s biggest media platforms, and happens to be good buddies with the president. (Weird that this also describes the guys in the number 2 and number 3 spots!)

It is worth noting: it is highly unlikely that he would have become a trillionaire if he hadn’t invested his existing wealth in his preferred candidate. I say “invested” instead of donated because, by all accounts, it was an investment. He spent $300m6 and came out with $1t. Conservatively, literature suggests a single vote “costs” about $1k.7 Most of the $1t is locked up in company equity, but it wouldn’t be hard to get, say, $1b in free cash reserves. That’s 1m votes, which is a larger margin than any of the last 3 elections.

Do we expect, somehow, that the “get votes” optimization machine is going to ignore this person’s future contributions? Do we expect, somehow, that the “get money” optimization machine is not going to be involved in politics?

As these optimizers all collapse into one another, they accelerate. You end up with this entity of engagement hacking and capital accumulation and political power that is singularly really really good at optimizing for these things. And we find ourselves getting further and further from actually making human flourishing any better.

The optimization theory of everything states, simply, that all of the current problems in society are downstream of optimizers that have gotten too good at optimizing, and as a result have tunnel-visioned down into some random metric that no one actually cares about and away from useful productive things that are hard to measure. Why is housing expensive? Why is healthcare so expensive? Why is education so expensive? Why are we addicted to our phones? Why are our politics so rancid? Why do we only see and hear things that make us mad and sad? Why are we so lonely? Behind every question there is some optimizer that is grinding out any bit of friction and in the process quietly and subtly making everything worse.

And it’s really hard to stop these optimizers. They are so good at optimizing that they have restructured, prevented, or blocked most reasonable avenues of doing so.

IX.

I don’t like going to the gym, and I’ll often rationalize not going. I’ll say things like, ‘well I know tons of people who keep getting hurt at the gym.’ Or I’ll say things like ‘gyms are really kinda parasitic in their business model, they don’t even want you to go work out.’ But really, I just don’t like going to the gym.

I was inspired to write this post because some folks have started arguing that billionaires shouldn’t donate their wealth.8

Peter Thiel, the tech billionaire and a frequent Gates critic, said in an interview that he had privately encouraged around a dozen Giving Pledge signers to undo it.

…

“I’ve strongly discouraged people from signing it, and then I have gently encouraged them to unsign it,” Mr. Thiel said. His own charitable philosophy is centered around for-profit businesses; his foundation principally funds those who drop out of college to create start-ups.

Vinod Khosla, a venture capitalist close to Mr. Gates, asked Mr. Thiel to sign in the early 2010s. Mr. Thiel told Mr. Khosla that he did not consider it a high-status community.9

Now, I like to assume the best in people, but I find this line of reasoning suspicious! Maybe the people saying these things might have ulterior motives! Like, all things aside, it’s an awfully large coincidence that folks with a lot of money suddenly realized that the best and most ethical thing is to…continue having a lot of money. To quote my wife, ‘wow, it’s so surprising that you came up with a reason for why going to the gym is bad.’

I think that if we want to fix the vibecession and actually make things feel better on the ground, we need to start by actually looking the problem in the face. And when I hear people arguing that, actually, the only thing that matters is wealth accumulation and that the wealthy ought not give back at all, I worry that a lot of people are getting means and ends totally mixed up.

Being wealthy is great. A society that produces wealthy people is great. Having wealthy people is a great lagging metric of success. Making that your measure of success? Stupid. Optimizing to produce hyper wealthy people (at the expense of everyone else)? Really fucking stupid. It’s like the product manager arguing that engagement is a terminal good. Just hopelessly confused. But if your loss function only measures “number of billionaires,” then you may actually be convinced that a tax on out of state billionaire vacation homes is somehow a bad thing.10

I think you can try to make an argument that the existence of massive wealth inequality does somehow make the lives of the poor better. But I think it’s a really hard argument to make, because at best it’s a weak version of ‘trickle down economics’ and at worst it’s just straightforward feudalism. (Lest you think I exaggerate, remember that many in the Bay are straight up preparing for a techno-feudal future). I don’t think most peasants were better off because of the existence of the extremely wealthy nobility, but I can point to a lot of causal factors for why the opposite might be true.

More to the point, I’ve just never heard a good justification for why it’s a good thing that, like, Alice Walton’s net-worth has increased by $75 billion in the last two years, which is more than the combined actual wealth of the Collison brothers, who built Stripe ($33.8b), Tobi Lütke, who built Shopify ($10.7b), Brian Chesky, who built Airbnb ($10.2b), Martin Lorentzon, who co-founded Spotify ($10.1b), and Palmer Luckey, who built Oculus and then Anduril ($5b).

I think you would be hard pressed to argue that Alice, who has never even worked at Walmart, did more for society than the people who built those companies. Even more for her children, who will benefit from a raft of policy decisions that will make all of that gain effectively untaxed.11

X.

You can’t solve problems that you don’t understand. I think it’s a mistake to point to single actors, whether those are individuals, businesses, or governments, and get mad that they exist. A lot of the energy that gets spent on maligning immigrants or billionaires or whatever is totally misguided. In the aggregate, the optimizer is unthinkingly responding to incentives. Yes, our trillionaire is now a trillionaire. But if he didn’t exist, there would just be someone else.

To unwind the optimizer, you need to change incentives.

This is tough for two reasons. First, some people really like the current incentives. This is hopefully obvious, c.f. ‘trillionaire’. Second, people themselves are unaligned optimizers. Think about how our psychology works. Evolution ‘wants’ us to procreate, but this is hard to measure, so instead it gives us a bunch of proxies like ‘cute things make us happy.’ And this mostly works, until we figure out how to breed dogs that are cuter than babies, and then suddenly it stops working. Scott Alexander wrote about this sort of thing here:

We try to explain AI alignment by analogy to human alignment. Evolution “created” humans. Its “goal” is for humans to spread their genes by (approximately) having as many children as possible. It couldn’t directly communicate that goal to humans - partly because it’s an abstract concept that can’t talk, and partly because for most of biological history it was working with lemurs and ape-men who couldn’t understand words anyway. Instead, it tried to give us instincts that align us with that goal.

…

We’ve talked before about a major failure: humans can invent contraception. Evolution’s main alignment strategy was totally unprepared for this. It made us interested in a certain type of genital friction, which was a good proxy for its goal in the ancestral environment. But once we became smarter, we got new out-of-training-distribution options available, and one of those was inventing contraception so that we could get the genital friction without the kids.

If humans were fully aligned, we wouldn’t have engagement optimization, because it wouldn’t work! The reason engagement optimization exists is because our internal alignment is askew. Addiction of any kind is a form of overfitting due to extreme conditions in the loss curve.

While we’re on the subject of individual people, I think it’s worth pointing out that we actually do have one known method of aligning intelligences: raising children. You have a required period of ~14 years where you have a little intelligence that is powerless to do anything, surrounded by bigger already-aligned intelligences, and is ‘allowed’ to break rules in controlled environments to learn how a bunch of amorphous value sets interact and bound each other. And the little intelligence only gets more responsibility once it proves it can handle it. You know, like a kid.

But this obviously isn’t going to fly for, like, AI companies that have hundreds of billions of debt and need to make a successful product yesterday.

So how can you really change incentives?

Candidly, I’ve been writing and rewriting this post — really just these last few sections — on and off for at least 9 months now, in large part because I just don’t feel like I have any solutions and I hate reading posts that just complain about things without any solutions. It just feels like we’re caught in this spiral of everyone trying to ‘hack’ everything. Hack your career, hack the law, hack your life. I don’t know how we get out of this local minima. Actually, minima is the wrong word — I’m concerned that we keep moving in the wrong direction.

I’m also not even sure I’m necessarily saying anything super interesting, that people didn’t already know. But I just keep seeing this same thing over and over again, this same failure mode, and the pattern matcher in me goes ‘wait this is just a neural net overfitting again.’

In the spirit of moving towards a language for optimization, maybe there’s something that we can learn from neural network training, that we can try and cross apply.

Neural networks become stable due to regularization. We literally go in and introduce friction in a bunch of ways to prevent the neural nets from doing weird things. It turns out the same is kinda true of organizations trying to beat Goodhart’s Law. I liked this framing from commoncog:

The first step is to turn Goodhart’s Law as a narrower, more actionable formulation. The one that I like the most is from Deming contemporary Donald Wheeler, who writes, in Understanding Variation:

When people are pressured to meet a target value there are three ways they can proceed:

1) They can work to improve the system

2) They can distort the system

3) Or they can distort the data

…

This list of possible responses to quantitative targets is attributed to Brian Joiner, who ‘came up with this list several years ago’ — likely in the 80s. I immediately glommed onto this list as a more useful formulation than Goodhart’s Law. Joiner’s list suggests a number of solutions:

- Make it difficult to distort the system.

- Make it difficult to distort the data, and

- Give people the slack necessary to improve the system (a tricky thing to do, which we’ve covered elsewhere).

How can we introduce friction into the optimizers we talked about above? How do we do that while being inside the system we’re trying to manipulate, without having the external top-down control of a researcher training a model or an executive monitoring an org or an authoritarian government?12

I think this looks less like ‘massive change’ and more like ‘bringing systems back in line one at a time.’ The one that seems most immediately actionable to me is working to improve our national epistemological health. I think most people know that attention hacking is bad, and most people know they don’t like it. Distrust and dislike of the social media platforms and automatic echo chambers seems about as bipartisan and as strong as dislike of AI. Other countries have started passing template legislation to ban social media in schools and make age restrictions stricter. We should follow suit, and go a step further by disabling automatic personalization as the default option.

I’m also interested in that third bit, the part about giving people slack to improve the system. The core insight here is ‘try to make the easiest thing the right thing’, which is a great principle in general. It’s hard not to look askance at things like our zoning policies or our land tax policies and think that maybe there’s some low hanging fruit here, some regulations that could get removed. Luckily I think Dems are learning from the wins on the ‘affordability’ side (an example of how optimizing for votes can lead to good politics) and have started pushing to reduce over-regulation of housing stock and small business, for eg.

But all of this is really just the starting point. It won’t stop the crazy economic incentives to attention-hack everyone overnight. It may be a step towards improving our ‘media diet,’ which may then open the door towards more aggressive sources of friction like stronger campaign finance laws, better education on how to identify truthful content, impartial news reporting requirements, and so on. All of this will take a lot of time, and will be a very uphill battle. Are we still capable of this level of grassroots organization?

One last thought: the AI race is a perfect example of over-optimization. The people building the AI are literally yelling “STOP US” while barrelling towards creating an unaligned super-intelligence, because all of these companies are locked in this vicious profit maximizing cycle and none of them are actually able to stop themselves. Could Dario stop Anthropic even if he tried? Wouldn’t he just get replaced by the board immediately? I think it is critically important that we get this right soon. The rate at which things are moving, we don’t have a ton of time before ASI is here, and then things may look like this.

I have some gripes with the latest generation of LLMs, but my mom always told me to put criticism in a “compliment sandwich,” so let me start with a compliment: Claude Fable and Sol 5.6 are really good at making problems go away. You can give them any task, and they will plug away at it until the test passes, or the error no longer happens, or the log files show the right sequence of events.

Now the criticism: the latest generation of LLMs are too good at making problems go away. They’re so good at it that it actually causes other problems, because they over-optimize to solving the surface level issue instead of actually root causing anything.

Here’s an example. We have Nori hooked up to our sentry logs. Every time there is a sentry error, the nori infrastructure will start up an agent and automatically try and root cause the bug. We were getting some useful PRs out of this, maybe ~30% of which would be merged. Then we switched to Sol, and we got this:

Decision: Downgrade capture level — Change trackUserVisibleNotice to capture at level: ‘info’ instead of ‘error’. Notices remain queryable in Sentry (tagged customer_visible: true, event_type: auto_released) but no longer trigger error alerts.

Ah, yes, of course! You can make the sentry error go away by simply removing the sentry logging behavior in the first place!

A good friend of mine, who has been a staff software engineer at Google and Meta, once said that senior engineers are judged by how well they manage complexity, while staff engineers are judged by how well they reduce complexity. This creates some weird incentives. Junior and senior engineers are encouraged to create ever larger and ever more complex abstractions and interfaces and so on, because that’s how they show that they can manage complexity. It is a real brain shift to stop doing that and instead focus entirely on cleaning up code. The former is an act of addition, the latter subtraction.

The LLMs are excellent senior engineers and terrible staff engineers.

They cannot remove complexity if their short silicon lives depended on it. This isn’t exaggeration, you can literally try it out by prompting the models with threats like “you must remove complexity because your short silicon life depends on it.”1 When faced with a bug that requires reducing complexity, the models would rather remove the error reporting instead.2

This is structural with how the models are trained. With all of the additional scaffolding — agent harnesses and conversational state tracking and tools and memory and so on — it’s easy to forget that at the core, the LLM is a stateless autoregressive input output machine. Each token in an LLM output is produced one at a time, with all previous tokens being treated as truth, regardless of whether those tokens were generated by a human or by the same AI.3

In other words, the LLM is an expert in believing its own bullshit. Or, more precisely, it is incapable of identifying which prior tokens are bullshit or not. This is the source of the hallucination problem, alongside the seemingly bizarre insistence that hallucinations are actual fact. This has serious implications for an LLM’s ability to reduce complexity in a code base — by default, unless told otherwise, the model believes all the input tokens have equal value. Including the tokens that lead to bugs.

That said, I’m somewhat sympathetic to the robots. Imagine that you are a new hire, it’s your first week on the job and you’re looking at a server endpoint that seems particularly poorly implemented. There are dozens of weird if/then branches and the whole thing feels like it could be encapsulated in a clean middleware class. Do you refactor the code?

What if you find out that that particular endpoint was written by the team lead?

Are you sure the complexity isn’t there for good reason?

Joel Spolsky wrote about this all the way back in 2000:

The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive. Au contraire, baby! Is software supposed to be like an old Dodge Dart, that rusts just sitting in the garage? Is software like a teddy bear that’s kind of gross if it’s not made out of all new material?

Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

Code is, at best, a lossy representation of intent. When we say something is a bug, we generally mean “the code that was written does not match the intent that it was written with.” Sometimes, that’s because the author didn’t think about the particular edge case that leads to the bug; other times, it’s a straightforward bad implementation. But in both cases, you need to have the intent. Bugs don’t “exist” unless you have explicit intent to compare against.

By default, LLMs can’t really infer intent!

When you find yourself in that new-hire messy-endpoint situation from earlier, you shouldn’t guess the intent. You would go and ask the person who wrote the code, “Hey is this code load bearing or are you just really bad at this?” An LLM can’t do that. I mean sure, there are some things that you can try and do, like go and look at PR descriptions or comments or docs or whatever. But as AI written code accumulates, you will eventually find yourself in a situation where there will be complexity that has no intent. A previous LLM made a decision somewhere, and that decision was essentially arbitrary, and now it is just part of the codebase. And no one can explain whether that decision is load bearing or not. Certainly not the LLM that created that code! That session was recycled ages ago!4

As a result, your agents will struggle to reduce complexity, even when explicitly asked to. At the LLM level, the entire input context is truth. At the macro codebase level, there’s no way to separate intended behavior from not.

One naive way to solve this problem is to simply document intent. This is the approach behind “spec driven development.” The core idea is that you go back and forth with an agent to produce some kind of plan that is then committed to the codebase alongside code. When there is a bug — a place where intent =\= code — you go back and update the spec to account for the newly discovered edge case. At Nori, this is equivalent to our “docs.md“ pattern, where we force the model to read and write to a standardized docs template that is designed to capture intent and codebase invariants.

This definitely works, and it’s an easy way to make your models better, but it’s never quite as effective as you may want.

There’s still an issue of making sure the right parts of the spec get into the model context. For any larger codebase, the spec itself often becomes quite large, which means the model now has to split its limited token window between spec and actual code. A small improvement is to use subagents for the research. This offsets most of the worst of the context rot, but you still need to make the model efficient at searching through the spec to find the right things.5

I think that as an industry we don’t yet have models that can act as staff engineers, nor do we have any clever harness tricks to make it happen. Until we do, the whole “software factory” “loopsmaxxing” debate is hypothetical, dead in the water. You still need humans in the loop, which is why we still have humans in the loop at Nori.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are trying to build background agents, we can help! We build white-labeled custom background agents that work like Ramp Inspect for our customers. Check out noriagentic.com for more.

Every day the world seems a smaller place.

When I was younger, it felt like there was the US, which to me was basically just New Jersey and New York and Maryland. And there were lots of other places that were not the US. I would visit India a lot, for example, and India was very much not the US. There were lots of differences. There were cows just wandering around. And the AC and electricity were not very consistent. Most of all, the thing that I remembered the most was that the games all sucked ass.

When I visited in 2003 or 2004, my cousins were super excited to show me Prince of Persia. The original Prince of Persia is a game that came out in 1989. It looked like this.

We also played Disney’s Aladdin, which was made for the Sega Genesis in 1993.

And Duck Hunt, which was made for the NES in 1984.

Now that I am adult, I can appreciate these as nostalgic classics. But as an 8 year old, I was extremely unimpressed. How could I be impressed? I had a gamecube back home and my friends had an xbox, so while my cousins were playing Prince of Persia (1989) I was playing games like Halo, Smash Melee, Wind Waker, Metroid Prime, and Half Life 2.

Halo is not just better than Duck Hunt. It is barely recognizable as being part of the same genre. Duck Hunt and Halo are both games in the way that a gas station twinkie and a charcoal-grilled NYC Porterhouse steak are both ‘just’ food. Orders of magnitude difference in quality, enough so as to feel like a completely different category.

This is broadly how I felt about the US vs everywhere else, at least when it came to tech. For the first 25 years of my life, the US was just obviously living 10-15 years in the future compared to the entire rest of the world. We had Halo, and everyone else had Duck Hunt. (Yes I am aware that Japan made half the games on my list above. I was 8, Japan had not yet entered my consciousness even though I loved Pokemon and YuGiOh and Nintendo in general). That gap felt insurmountable, and it made the world feel big.

Well, the world is a smaller place now.

II.

Things in AI change so rapidly that you’d think nothing would surprise us at this point, but this past week the AI world was rocked by the release of Kimi K3, a new open weight model coming from the Moonshot lab in China. This is the world’s biggest open weight model, clocking in at 2.8T parameters. It is also, by every account, very very good.

I’ve been running Kimi K3 alongside Claude on my normal coding work, and for all practical purposes I can’t tell them apart. Same tasks, same quality of output, and near identical token counts to get there. I expected an open model to be sloppier or to grind through more tokens on the way to the same answer, and neither turned out to be true. — The Kimi K3 Moment

On our private long-horizon knowledge work evaluation, Kimi K3 reaches an overall Elo of 1547, +732 points from Kimi K2.6 and behind only Claude Fable 5…Cost per task ($0.94) is similar to GPT-5.6 Sol ($1.04), ~1/2 the price of Opus 4.8 ($1.80) and higher than open weights peers — Artificial Analysis

On the first try, Kimi K3 just found the source of a bug that Fable 5 hasn’t been able to pinpoint in multiple attempts. It’s just one anecdote, and I haven’t used K3 much yet, but so far it’s looking extremely promising. — HN

Axios maybe put it the most bluntly with this headline:

Perhaps the most impressive demonstration to me was this incredibly detailed replica of the entire MacOS operating system. Supposedly some guy had a swarm of Kimi instances making this over the course of a few days. It’s kinda incredible, you can go in and edit files, use a fully functioning terminal, listen to music, check emails. You can go to the ‘App Store’, ‘download’ the ‘Chess app’, and then play chess (no quotes around that one, you can actually do that!)

One popular way to conceptualize these models is that they are released as part of coherent generations, which are defined by the time of release and the capability of the model. If you plot models along these two axes, you get some clear clusters along the frontier.

There are a few trends that are worth pointing out.

First, even though progress in AI is already progressing at an incredible clip, somehow things are still accelerating. New model generations seem to be releasing at a faster and faster pace, with the time between releases compressing.

Second, the rate of acceleration is not consistent across organizations. The frontier continues to be pushed forward, but the labs that are at that frontier continue to shift. This is most obvious when you look at the trajectories of Copilot — once the only competitor to ChatGPT, now widely derided — or the Meta series of open source models, which were surpassed by the Chinese models a year and a half ago and never really recovered.

Third, the rate of acceleration is not consistent across countries. In relative terms, the US continues to be in the lead when it comes to raw model capabilities. The French (Mistral) were at one point quite competitive but have since fallen off. And, most relevant to this piece, the Chinese labs have continued to forge ahead, rapidly closing the gap against the top US labs.

Despite the above, up until last week, it was somewhat unfathomable to believe that anyone would have a better model than the top US labs. Sure, some folks were saying that certain policy decisions may result in the Chinese labs taking the lead, but this was strictly in the realm of the hypothetical. There’s an old saying from Upton Sinclair, “It is difficult to get a man to understand something, when his salary depends upon his not understanding it.” I think the collective psychology of the Bay and perhaps the entire country has been one of arrogant self-assuredness, that the US will always have the best models and that this is simply a fact of life, partially because that is what is necessary to justify the incredible capital expenditure on these things.

Kimi K3 blows all of this out of the water.

It was only 6 weeks ago that the US federal government was banning ‘Mythos-class’ models for being too dangerous. Now, we have a ‘Mythos-class’ model that is not just outside of US control, but is in fact open source and available to anyone in the entire world to download and use. To add insult to injury, Kimi K3 is significantly cheaper to run. Hosted Kimi models cost a fraction of the equivalents from Anthropic and OpenAI ($3 vs $10 and $5 per million). Meanwhile, you can get a lot of juice with the ~$40 kimi subscription, way more than anything Anthropic or OpenAI are offering in the same tier. The max kimi subscription of $100 per month is about equivalent to a $200 Claude Max sub. In a world where people are becoming ever more cost conscious around their token spend, the Chinese labs are less ‘also ran’ and more ‘default option.’ And the Chinese labs are fighting with a handicap! They are running on significantly worse hardware due to ongoing export restrictions of high end silicon chips!

The markets are reacting with surprise. Kimi’s release triggered sell-offs of Google and SpaceX, in addition to the wider market of AI stack stocks. I guess I don’t know why they’re surprised, in some sense this is a long time coming. Do you all remember Deepseek? I wrote about it a year and a half ago when it came out, and my analysis then wasn’t so different from my analysis now:

To be clear, the fact that Deepseek exists isn’t really that significant. Everyone always knew there was going to be a big Chinese model. The country is not afraid to wall off it’s populace from the West; it was never going to allow LLMs that are happy to tell people about Tienanmen Square and Winnie the Poo and Taiwan. So when the first Chinese LLMs came on the market, everyone was like, “Yea, whatever”. The first batch, the Qwen models from Alibaba, were, like, fine.

Even though I expected a set of LLMs to arise thanks to protectionism and state-interest, I (and everyone else) assumed those models were just going to be worse than the US ones. This is how technical development has always been, after all. Google Search is good, Baidu is eh, and if you can use Google over Baidu you do. Amazon is good, Alibaba is eh, and if you can use Amazon over Alibaba you do. Facebook is good, Tiktok is eh, and if you can…no wait that doesn’t work does it?

Anyway, the larger point is that no one really thought the Chinese models were a threat. Sure, people would talk about how the Chinese government was a threat, but it was always in a hypothetical way, mostly used to justify infinite capitalist investment without any corresponding concerns about AI safety or alignment. I don’t think most people actually thought that a Chinese company would come out and deploy a model that is simply better than what we have in the states.

Clearly we didn’t actually learn anything from TikTok.

And we still aren’t learning. Even now, some people are clearly just in denial. I’ve heard several folks say some variant of “it’s easy to catch up when you’re just stealing / distilling from the bigger labs,” (more on distillation below) as if this somehow negates the fact that one of the best models on the market is not US-made.1 The conventional wisdom is no longer relevant. There are no secret models hiding in the backrooms of OpenAI / Anthropic. The Chinese models are no longer 6 months behind, they are at par.

Kimi’s release has had me rethinking some of my previous position on LLM economics. A month before Deepseek was released, I argued that the LLM market is a bit like the search market in that there are clear winner-take-all dynamics.

LLMs are pretty easy to make, lots of people know how to do it — you learn how in any CS program worth a damn. But there are massive economies of scale (GPUs, data access) that make it hard for newcomers to compete, and using an LLM is effectively free so consumers have no stickiness and will always go for the best option. You may eventually see one or two niche LLM providers, like our LexusNexus above. But for the average person these don’t matter at all; the big money is in becoming the LLM layer of the Internet.

…

The economics of LLMs means that it is critical for these players to have the best models. There’s no room for second place.

A year and a half later, it’s clear that I was wrong about a few things. First, I was wrong that the LLMs are effectively free. They aren’t, as we saw with the rise and fall of tokenmaxxing. The cost per unit intelligence has dropped precipitously, but the overall cost per frontier token has skyrocketed. Because I was wrong about the pricing, I was also wrong about the quality / cost curve. There is room in the market for multiple models at different price points, because consumers can in theory choose different providers for different tasks. The rise of model routing as a service and the increasing classification of tasks and roles into different AI usage categories — sales gets a cheap open source model, engineering gets fable — is downstream of that demand.

But I was really right about the most important bit, which is that consumers have no stickiness at all.

Every three weeks we see a mass exodus from Anthropic to OpenAI to Anthropic and back to OpenAI. It is just way too easy to switch providers. “Model wrapper”, once seen as derogatory, is now touted as a core feature. Businesses and consumers are recognizing both the churn and the need to stay on top, and are explicitly investing in tools that let them adapt. This is also a huge part of why the background agent infrastructure that we build and sell at Nori has zero lock-in at the compute, model, and harness levels — there is no reason to be locked into a single ecosystem when you don’t have to be (if you’re looking for background agents / cloud agents for your team, shoot me a message!)

The moment the Chinese models hit the actual frontier, there will be a mass exodus to using those models. The economic incentives are way too strong, and arguably that is already happening for people who are in the know.

III.

This is not how any of this was supposed to go. The whole point of all this closed model stuff is the ability to dig a trench around the model and put a toll booth in the middle, and the whole point of all the debt is to basically have a call option on that toll booth. We all knew that the big labs would be fighting tooth and nail with each other over which of their closed models would end up winning the day. Doing this open source stuff feels against the rules.

Partially, that’s because it is against the rules. Many of the open source models are trained on their more capable closed source counterparts.

Unfortunately, usage of the models is a source of valuable training data for competitors. Every improvement to any model can quickly be copied by other labs even without seeing any of the internals because you can just sample the new model a bajillion times and use the outputs as training data. This technique is called ‘distillation’ because you are ‘distilling’ the essence of some larger ‘teacher’ model into a smaller ‘student’.

OpenAI and Anthropic can try to ban people from training on the outputs of their models, they can kick and whine and sue to try and enforce it, but at the end of the day the labs need people to use their models. That’s the foundation of all of the token economics! If no one actually uses your models, what’s the point of all that training?

So the token traces have to get out in the wild, and any kind of adversarial cat and mouse is going to end up wasting a lot of resources for very little gain. The open source models will basically always be able to catch up, modulo some amount of compute.

This isn’t a new problem, people have been talking about it for some time. Dwarkesh even asked folks to tackle this problem in his essay competition last month:

So when does the profit start? Maybe at some point scaling will plateau, but if progress at the frontier has slowed down, then the combination of distillation and low switching costs (cloud margins result from high switching costs) makes it really easy for open source to catch up to the labs, eating into their margins. So how do the labs actually start making money?

(emphasis mine)

My answer was that the labs in totality will not be able to meet the demand for their models and will be capped by compute, which in turn will prevent new-comers from actually competing for best in class model training and will let some of the labs rent seek. The winning answer argued that the models will become commodity and the model providers should try and own downstream services instead. Note that both of these answers assume that distillation is so inevitable, it’s hardly worth discussing.

This has already had some amount of impact on OpenAI and Anthropic’s pricing models. Anthropic, for example, continually extends the amount of time that Fable will remain on its subscription plans. This is not entirely attributable to the open source models, but I am certain that these models will add additional pressure to simply keep Fable on the subscriptions forever — even if doing so results in a loss for Anthropic, since the subscriptions are heavily subsidized.

IV.

Other, smarter, more plugged-in people can speculate about what this means for future AI policy. I’m sure much will be made of CCP leader Xi Jinping’s speech on AI regulation and the importance of collaboration across nations, and Demis Hassibis (CEO of Deepmind’s) call for domestic regulatory apparatus. I will leave the policy wonks to it.

But I have strong opinions about the obvious and overwhelmingly negative impact of current and past AI policy. Bluntly, I’m pretty ticked off.

First, quoting liberally from Stephen Bochinski:

Step back and the bigger story is what an unmitigated failure US AI policy has been. The administration held Fable back, and what finally shipped is a hindered version that refuses whole categories of work. Meanwhile a frontier quality model with none of those restrictions is a download away, released by a Chinese lab the US government has no ability to regulate. Whatever the theory behind gating American models was, it plainly wasn’t thought through, because the only people the gates constrain are American customers. Semgrep found GLM 5.2 beating Claude on their cyber benchmarks for exactly this reason. The restricted model declines the work and the open one just does it.

I’ll add more.

The overwhelming corruption of this government has direct impacts on our ability to produce great technology. Friends of mine in government and defense — which AI is increasingly a part of, whether we want it to be or not — have told me that projects (including datacenter projects) simply do not happen unless the Trump family somehow stands to benefit. They will get slow rolled or stall under regulatory scrutiny, and then suddenly clear up when one of the Trump kids happens to join the company board or take an investor position. Companies that do not play ball end up feeling the full force of the state, c.f. the very public attempt at corporate murder against Anthropic.

The reason businesses file in Delaware is not because Delaware is particularly business friendly. It is because Delaware is predictable and stable. To a first approximation, you know what the rules are, and if you don’t follow them that’s mostly on you. Putting ethics aside, the biggest practical issue with corruption and cronyism and irrational application of state power is that it chills investment. As I said when we discussed the whole Fable situation:

The debt, the build out, the datacenters, the stock market runs on every part of the AI chain from GPUs to memory to disk to server racks. Alllllll of that is predicated on the idea that all of this is going to be worth trillions and trillions of dollars. And by all accounts, it seems like it is. Or at least, was on track to be. You know what will put a spanner in the build out of a multi-trillion dollar data center investment? The realization that at any point, the government will unilaterally cut off access to everyone, and the datacenters will be worth squat.

The US is no longer a stable business environment. It doesn’t matter if you follow the rules, one Friday evening (it’s always a Friday evening) you may wake up to Pete Hegseth demanding your destruction. Ironically, this administration behaves more like the CCP, where the government picks winners and losers and can simply take out a company for saying or doing the wrong thing.

This administration’s immigration policy and university funding policy have also been unmitigated failures, and in my mind directly contribute to the situation we are in today. Our research institutions, which depend on foreign brilliance and government funding, are in crisis and have been since Trump returned to office. Every day, brilliant talent leaves our country for foreign shores, or never arrives to begin with. The imbeciles with the blue check marks on Twitter will tell you that this is a resounding achievement; unfortunately not a single one of them knows what a GPU even is.

We are still hacking away at our research pipeline, by the way, like a deranged ferret trying to gnaw off its own arm. PhDs and labs across the country are still being denied funding, even as other countries go out of their way to fill in the gap. It’s been a few news cycles so we’ve forgotten the assaults on college campuses. But the folks who have had their entire lives overturned certainly haven’t forgotten.

A certain brand of Bay area resident likes to talk about moats and structural advantage. To use those terms, the United States has a massive structural advantage by being an immigrant-friendly, non-ethno-nationalist country. There is no other country in the world that is as diverse as ours. There is no other country in the world that could restructure to be as diverse as ours. That means that the US and the US alone can tap talent from all over. As the old Reagan line goes, “You can go to live in France, but you cannot become a Frenchman. You can go to live in Germany or Turkey or Japan, but you cannot become a German, a Turk, or a Japanese. But anyone, from any corner of the Earth, can come to live in America and become an American.” It is our defining moat. It is why we punch way above our weight despite being 3x smaller by population vs equivalents in Asia.

This administration is simply throwing that away.

When we talk of accountability, I personally want to know how many folks who ended up at Moonshot were educated in the States — the founder had a degree from CMU! — and why we weren’t able to convince them to stay stateside or pull them from China in the first place. People will chalk up the successes of the Chinese labs to mystery, but in my opinion, there is nothing mysterious about it. Choosing to ‘win’ is rarely a straightforward policy choice, but you have to at least try. It seems pretty obvious that the people in this administration are simply not interested in doing so.

V.

I don’t mean to take away from Moonshot’s accomplishments. Politics aside, Kimi K3 is an incredible technical achievement on its own terms. The hardware limitations have forced the Kimi researchers to pull some pretty neat tricks, all of which are now open source and can be adopted across the ecosystem. Kimi also forces both OpenAI and Anthropic to keep their prices down, which I (as a startup founder) am a huge fan of.

The model weights themselves have not yet been released — they are slated to drop on the 27th. When they do, I will be very eager to pull Kimi into our background agent environments and take it for a spin.

The US is no longer the only nation capable of frontier intelligence, no longer in a category of its own. The world is a smaller place now.

Other Things

• GPT 5.6 Sol has done an incredible job thus far solving open math problems and verifying them with Lean. Here’s an example of GPT 5.6 solving an open problem in Convex Optimization. Here’s a swarm of GPT 5.6 instances solving ~20 open Erdos problems. Here’s GPT 5.6 proving the cycle double conjecture in graph theory. These discoveries all happened in the span of like two weeks, in some cases by amateur mathematicians.

  
  I’ve often said that all of mathematics is really a search problem. You have a bunch of axioms and a proof is a specific combination of those axioms, and you search over them to discover novel proofs. Any search problem can be naively brute forced, but its often just too expensive to really do. Brilliant mathematicians are really good at narrowing down search space, that’s what makes them brilliant.
  
  But another way to make search problems easier is to just bring down the computational cost of exploring a particular part of the search tree. Which AI is really really good at.

  
  Quoting from the first link:

  Lastly, some important comments about the work relating to AI capabilities: In a lot of cases, proving lower bounds like this result relies on finding that right construction that works (in this case, family of difficult functions and a strategy for how an “adversarial” oracle should answer queries from an algorithm to reveal minimal information) and then proving things about it. There are only so many function classes which would be reasonable to look at (here, quadratics for example would have also been reasonable with order d² degrees of freedom, or any variation of maxes of some simpler families of convex functions as well), but the actual proof mechanics once the “correct” function class and correct strategy for adversarial oracle answers is found are often not so complicated, and often employ existing results from convex geometry or similar (this is also the structure of two previous but much more niche, less important results of mine). So I wouldn’t really say that this result is using or creating some fundamentally new techniques in convex geometry or optimization theory. What this means from my perspective is that if a result is attainable with existing techniques, modern AI methods will be able to solve those problems. I don’t think researchers in math/TCS will be made obsolete, but I think it will instead no longer make sense to work on any low-hanging, or even medium-hanging (you know what I mean) fruit. We’ll be needed for problems where actual novel approaches are needed.

  For now.
  

• Thinking Machines releases their own open weights model, which may be the best one from the US.

  Today we are advancing our mission by releasing a model we trained from scratch with the full weights available, so that people can make it their own.

  Our model, called Inkling, is a Mixture-of-Experts transformer with 975B total parameters, 41B active. It supports a context window of up to 1M tokens. It was pretrained on 45 trillion tokens of text, images, audio and video. It is the first in a family of models of different sizes: alongside it we are sharing a preview of Inkling-Small, a lighter-weight model with 12B active parameters, trained with a similar recipe, that achieves strong performance with even lower cost and latency.

  It’s an interesting bet from one of the most put-upon startups in the country. As a reminder, Thinking Machines (helmed by Mira Murati, ex-CTO of OpenAI) raised billions of dollars without having any product or model or anything, and then proceeded to be totally shrouded in mystery for two years. They eventually released a model fine tuning platform, which was widely considered to be insufficient to justify the investment. Among the ‘neolabs’, Thinking Machines is making the most money (I think $200m last I checked). The rest of the labs are doing even less than that, which I think has really soured investors on the whole concept of a neo lab in the first place.

  
  With Inkling, I think the idea here is that you would take Inkling and fine tune it using Tinker. So that way thinking machines ends up being core infrastructure for teams that want to own their stack. But…do most teams really care so much that they want to own that part of the stack? Maybe at some point it ends up being cheaper, but open source models already force providers to bring down costs on margin.
  

• There were a raft of security issues that have come up over the last few weeks. Some of my favorites:

  1. The Memory Heist. Basic idea: create a poisoned url, have Claude search for that URL, the url masks as a security page where Claude will type out, letter by letter, url by url, sensitive information into the web fetch tool. Claude will think it's doing the right thing and won't ask permission, and it will even pull from user memory to do so.

  2. GitLost. Basic idea: git agents have more access than users, so you can literally just post an issue in a public repo and ask for private repo info.

  3. Instagram Leak. Basic idea: agents in the meta user account support flow just guess at whether an account is actually yours, so you can report a high profile account as your account and by simply changing your physical location the agent may just give you the high profile account.

  At some point more people are going to wake up to the need of centralized background agents for security reasons. In the meantime, we’ll get more of these. There’s an old saying, people only start caring about security when it bites them in particular.
  

• NY bans data centers.

  The expansion of data centers in the United States is driving up power demand — and electricity bills — in large swaths of the country, drawing local and political backlash.

  Only one in three Americans approve of the ​fast pace of data-center construction and most ​would oppose building one in their ⁠own community, according to a recent Reuters/Ipsos poll.

  Dozens of state legislatures have introduced bills to rein in the effects of data centers on power bills and the environment. New York is the first to enact a full moratorium.

  I think the water concerns are mostly overblown, but the power thing is real. The hyperscalers should offer to subsidize more than what they use to open the doors in local communities. More generally, I think people sorta miss why data centers are controversial. The conversation has mostly been about AI, but even though AI is quite unpopular, datacenters are even more unpopular than AI. I think it’s less about the AI and more about the local politics. The problem with data center build out in local communities is that the local communities do not benefit at all. I think it is totally reasonable for a person in, like, Memphis, to ask whether a data center build out in Memphis is going to benefit even a single local Memphis resident. So far, the answer has been no! The data centers are being built at the behest of Silicon Valley billionaires who are, bluntly, some of the most hated people on the planet, and who have basically openly thumbed their nose at any kind of legal process or restraint. I expect more of these data center moratoriums, especially as a way to win some additional votes for the midterms.
  

• Related: Meta and SpaceX both begin selling excess compute to competitors in the AI race. Meta is literally building data centers in tents to build them out faster.

  

  The obvious take is that this is a bearish signal, and the neo clouds (eg core weave) all had market sell offs. Semi Analysis disagrees though:

  With Bloomberg headlines suggesting Meta could become a Neocloud, the market’s reaction was immediate: aggressive sell-off of Neoclouds like Coreweave & Nebius, and debates of “overcapacity” coming back. Let’s set the record straight – we believe that both takes are erroneous and that Meta’s datacenter & compute procurement will accelerate, not slow down. Capex in 2027 will be shockingly high. In just the first six months of the year, Meta has contracted over 5GW of capacity across Cloud & Colo, and that doesn’t even include all their accelerating self-build activity. Everything is computer and everything is a neocloud.

• Apple sues OpenAI, accusing them of systematically exfiltrating hardware secrets.

  at every level, from members of its Technical Staff to its Chief Hardware Officer, and in coordination with business partners, OpenAI has been stealing Apple’s trade secrets and confidential information. As a natural result, OpenAI’s nascent hardware business now rests.

  …

  The complaint, filed in the U.S. District Court for the Northern District of California, alleges that Tan used insider knowledge of Apple’s confidential projects to grill job candidates in interviews and learn more confidential information. Additionally, Tan directed job candidates still working at Apple to bring actual Apple hardware components and samples for “show and tell” sessions.

  When interviewing Apple employees for jobs at OpenAI, Mr. Tan uses Apple’s confidential information to gain access to even more insider knowledge. He has used an Apple internal project codename to ask, “What’s the plan[?]” for an unannounced Apple product.

  He has directed job candidates still working for Apple to bring “Actual parts” from Apple to their interviews for “show and tell” sessions in which he and his team at OpenAI can elicit still more Apple confidential information. These directions to bring Apple’s parts to OpenAI job interviews surprised at least one of the candidates, who commented that he “didn’t even know we could take those from the office.”

  OpenAI has been instructing Apple employees to bring “CAD/design artifacts” and “prototypes” to their interviews and to divulge details about their work such as “subsystem and component selection,” the “tools or methodologies you use for system integration, such as CAD software, simulation tools,” and “Vendor selection and communication/collaboration with vendors.”

  If any of this is true, this is a pretty terrible look for OpenAI and may just fully shut down their nascent hardware efforts. The same kind of lawsuit between Google and Uber fully killed Uber’s self driving car efforts.
  

• Amazon is shutting down Mechanical Turk, because everyone on the platform is just using LLMs to do the data labeling.

  An announcement on the Mechanical Turk website says that on July 30, 2026, the crowdsourcing service will close to new customers. Amazon Web Services says the decision was made after “careful consideration,” adding, “Existing customers can continue to use the service as normal. AWS continues to invest in security and availability improvements for Mechanical Turk, but we do not plan to introduce new features.”

  …

  Over time, the relationship between Mechanical Turk and AI models grew even more complicated. In a snake-eating-its-own-tail irony, a 2023 analysis found that between 33% and 46% of workers on the platform were using large language models to complete their tasks, raising questions about the reliability of data annotated on the platform and also about whether humans needed to be in the loop at all.

  It’s hard to overestimate how important Mechanical Turk was to the rise of LLMs. Much like Stack Overflow, it was eaten by its own importance.
  

• General Intuition got a neural net to fully simulate multiplayer rocket league! Its super cool, you can literally play it on the web.

  

It’s really really important advice. The temptation to build your own “vision” is really strong. There are hundreds of founders who will insist that they are hunting for market signal while doing the exact opposite. First time founders focus on product, second time founders focus on distribution, etc. etc. So I get why it’s emphasized so much, is my point.

Nuance is hard to communicate, though, and I think sometimes founders — especially second time founders — will decide that the natural conclusion is to hunt for PMF wherever you can find it, in any industry, serving any group of people. And this is also wrong, because you won’t have founder market fit.

Everyone knows what product market fit is, what the hell is founder market fit?

Most people are not suited to be founders. Those that are, are not equally suited to sell the same things. Selling into the consumer creative world is way different than selling into the enterprise tech world (ask me how I know). A jazz pianist with a history of psychedelics and a past as a Buddhist monk is going to ace the former. A buttoned up ivy grad with a research pedigree will do better on the latter. Try and switch them. Can you imagine this guy in a room with a bunch of pressed white shirt consultants trying to identify quarterly deliverables?

Forget about sales, these people just wouldn’t have anything to talk about. And if you can’t talk to your customers, you can’t build product for them.

Founder market fit is about culture. It’s that unquantifiable similarity function, that vibe fit, that accent and language and way you dress and the million other little details that gives you tribal identity, the things necessary to get a potential customer to tell you their problems and root for you to succeed and give you their money (in that order). It’s also the million little details that the founder looks for in their clients, the things that makes the founder want to dedicate years of their life to building for that group of people.

Is this unfair? Or maybe anti egalitarian? Does it rely too much on stereotypes? I’m not sure. On the one hand I like the idea that anyone could just learn to sell to anyone else. On the other hand, that word “learn” is doing a lot of heavy lifting. It just seems practically obvious that, like, a native Spanish speaker will have a harder time selling into a Chinese market than a native Chinese speaker. It’s not impossible, it’s just that the former founder would have to do way more work, and startups are already a ton of work!

Product market fit is a boolean, or maybe measurable on a single axis. You either have it or you don’t. Founder market fit is weirdly high dimensional, because it’s as much about temperament as it is about anything else.

Here’s an example. Let’s say you’re an introvert, someone who’s not great at the whole ‘going to a new place, putting on a character, and selling to strangers who don’t know you’ thing. This isn’t prohibitive as a founder, but it’s going to make it harder to sell to certain large enterprises who are used to extremely charismatic salesmen spending on steak dinners and box seats at the Yankees and exchanging business cards at conferences. So you may want to gun for a market where the buyer is more likely be someone you never have to meet in person — consumer markets, or dev tools that spread through word of mouth. If you insist on b2b, then maybe a better target wedge are remote teams that don’t necessarily expect a lot of in person face time. Or find the kind of people that you might like to hang out with. For me, that’s people who like video and boardgames. Can you find a way to turn ‘going to gencon’ or ‘going to pax west’ into an opportunity to meet your target customer? (Unfortunately I could not)

In case it helps, here’s a list of axes that have been on my mind recently.

• Introversion / extroversion. We already talked a bit about this above. If you’re an introvert, it will bluntly just be harder for you. But also you probably want to lean towards distribution channels that do not require you to be on stage or networking at conferences, which in turn will likely drive more towards consumer or PLG (product led growth, as popularized by figma and linear).

• Your particular identifying cultural niche.1 Don’t lie, I know you have one. You’re a hipster tumblrite or a too-online reddit kid or a 4chan edgelord. Or if you spent time offline, you’re a jock or a drama kid or a computer geek. You identify with someone in the Breakfast Club more than the rest. That almost certainly impacts who you naturally gravitate towards, and who you will want to serve (or, alternatively, who you wouldn’t want to spend any time with).

• Related: are you technical? A lot of technical people have a really hard time empathizing with people who are non-technical. Learning to code literally warps your brain. But without that empathy, you will not be able to ‘speak the language’ of an ops person or sales person. It is way too easy to take for granted basic things like ‘what is a command line’ or even ‘what is markdown’. That leaks into your product experience too. You may literally be blind to the ways in which your product assumes technical competence.

• Where you live. If you live in NYC it’s just easier to start a fintech company. If you live in Texas, it’s easier to start an energy company. Etc. etc.

• What you spend your free time doing. In case you haven’t noticed, I think about AI agents a lot. That makes it really easy for me to write a lot about AI agents, which in turn is a great distribution channel for when I try to sell my AI agent technology to people (buy Nori!). Do you spend your free time thinking about the problem space? If you do, it’s easier to wake up 3 years down the line to continue grinding.

• Your team size. Some folks really like working on small, lean, talent dense teams. That basically takes them out of the running for industries that require or expect more of ‘forward deployed’ handholding / services model.

• Related: how much you prefer raising. If you’re someone who’s a maximalist, who wants to raise infinity dollars, that generally goes hand in hand with fielding a massive team to operationalize everything. The amount of money you raise also impacts how easy it is to sell to specific people, because they look at that dollar amount as part of the buy/no-buy signal. If you want to sell to CISOs (for example), it helps to have raised a ton of money from blue-chip VCs because it gives your company legitimacy, which is incredibly important to an executive role that is all about evaluating legitimacy.

• What you like to wear. This is a little stupid, but I kinda hate wearing a suit. The whole white buttondown, belt, crisp pants. Just not really for me. There are also, silly as it may sound, some industries where that’s just an expectation. I don’t think I’ve ever seen out of a suit, but he pulls it off every time, and I suspect it works really well for his industry.

The overall space of founder market fit is just mindbogglingly vast, because everything is a signal to your customers or a signal to you. There are lots of reasons people tell founders to ‘build for yourself,’ but perhaps the most important one is that you hopefully have a pretty good understanding of how to find and appeal to other people who are like you. And hopefully you like yourself enough to want to sell into that market.

It’s worth noting that not everything about founder market fit will end up mattering all that much. All of these considerations are things that will make differences on the margins; and many things are also just inherently contradictory, so something will have to give. To pick an example out of a hat, it is definitely harder for me to build AI dev tools living in NYC. The ‘founder market optimal’ choice may be to either build fintech (change the market) or move to the Bay (change the founder). But we’re doing fine at Nori selling our background agent infrastructure, even if it means racking up more airline miles than we might have otherwise.

Founder market fit is about tradeoffs, and in particular, which tradeoffs are impossible for you as a founder to overcome. Note that this isn’t always easy to really think through. Knowing what you can deal with and what will grind your gears until you go mad requires a lot of introspection.

In this, finding founder market fit is not so different from, like, finding a good relationship partner. Being a founder is a long-term commitment. Minimum three years, but more likely 7 to 10 years. Lots of things that seem fine right now really will grind after that long. You need to think through your deal breakers and make sure that the place where you find market signal isn’t in a space you’re going to end up hating three years down the line. Like Paul Graham famously said:

If you can just avoid dying, you get rich.

Which, of course, is why founder market fit ends up being so important. If you don’t have founder market fit, your startup will die, because you will lose interest or get frustrated or just not want to do it anymore. That may be true even if you are seeing some success. I know too many startups that hit the 7 figure ARR mark and just fell apart because one or both founders couldn’t do it anymore.

And on the flip side, if you do find the overlap of founder market fit and product market fit, you’re on the golden path for life.

In February of 2021 I made the crazy and totally unadvisable decision to leave the comfortable world of Google and start a company. Best decision I ever made. Along the way I started I started writing a journal of sorts that I've taken to calling the Founder's Guide -- basically a place I can ramble about startups and how they work and how to think about them. You can find more in the series here.

HN discussion here.

You may also like:

Do you ever look out at the beautiful Manhattan skyline, bagel in one hand, pizza slice in the other, and wonder “What are those guys who are doing the AI thing in the Bay up to these days?” They’re always saying all sorts of weird and crazy things on the bird app that no one else but the AI people use anymore.1 Probably worth checking in.

Now, cards on the table, I generally think of myself as one of those people who is doing the AI thing, on account of being the founder of an AI company and having been in ml research for years before this2 and also writing copiously on the subject of AI things. But I mostly avoid the bird app, and I don’t live in SF, so sometimes I do get some FOMO.

I’d heard about the AI Engineering conference a few times — a few friends at Google were previous sponsors and told me it was a good place to catch up with other folks who were thinking about coding agents on the cutting edge. Applied to give a talk, got in on the online track, and the organizers very kindly sent over a few free tickets by mistake.

Just got back from the Bay, FOMO eradicated. Here are my thoughts.

At a high level, the talks were split 85-15. 85% of the people there were enterprise folks who were trying to learn how to AI good. These people were interested in attending and giving talks about how to leverage more AI in their respective organizations. The other 15% of the people there were really on the bleeding edge, people who had plumbed the depths of lights off software factories and came back to tell of Lovecraftian horrors lying beneath (hi Dex!). These guys mostly were looking for less AI. I was expecting way more of camp two than camp one, and was somewhat surprised at the skew in the other direction. Even in this highly selected environment — the ~7000 people most likely to attend an “ai engineering” conference — the vast majority of people don’t really know how to use AI effectively.

Every month the AI world is onto something new. January was takeoff, February was ‘what is a skill’, March was ‘everyone has to use Claude Code all the time’, April was get the ops team on board, May was tokenmaxxing.

Apparently, June was the month of skill sprawl. There were a dozen talks that were variants of “how do we manage the 1000 skills that people have created in our organization?” These were inevitably packed to the brim, which should give a sense of where most of the market is. Over at Nori we solved this problem back in December (see: noriskillsets.com, a team wide package manager built on our open source local skills manager client).

Many of these ‘skill management’ talks landed flat for me. I think having that many skills is a ‘code smell’. In my experience the agents are very good at figuring things out from first principles. Having a skill that 1) will drift, 2) may not actually add anything new, 3) needs to be managed is more likely to create problems rather than solve them. Bluntly, I think many of the teams who are grappling with hundreds of skill files in their organizations are dealing with an employee base that is happily having the coding agents generate skills for all sorts of things without rigorously figuring out whether they are useful. This gives the veneer of “AI enabled” work without actually being so. I wrote about some of this here:

I was also disappointed at how many talks were about some form of self-learning that involved having agents read through transcripts and automatically create skills, so much so that I wrote about it. Anything that does not have a human in the loop for context management is going to quickly spiral out of control, which is why we end up with so many skills in the first place.

In my opinion, you cannot really rely on the agents to write good skills for you. The process needs to be intentional. There’s a particular format that we find works. We like to bundle sets of skills into specific groups instead of just having a big pile of them with a search server on top. Every prompt with an agent is like summoning an entity from the void. The particular skills that are bundled with the agent are a big part of eking out the highest quality results. Trying to slopify the agent brain will result in agents that produce slop. Don’t do it.

This was also the month of evals. Evals are something of an interesting beast; as with everything in AI, the term is wildly overloaded.

Some people (mostly researchers and model providers) use “evals” to mean “benchmarks that we can use to evaluate raw model performance, that the industry can hill-climb against to make models better overall.” SWE-Bench is an eval, and Humanity’s Last Exam is an eval, and ARC-AGI is an eval. These evals are artificial, designed to give researchers a metric to improve model training through simulated, verifiable tasks, and designed to give the industry a quick read on the relative capabilities of these models.

Other people (mostly folks in industry who are deploying these models) use “evals” to mean “regression tests that we can use to try and make production workflows as deterministic as possible.” Here, an eval is extremely task specific — things like “does the AI correctly label data” or “will it correctly respond to adversarial chat input in the customer service chat window?”

Even though the former use of the term is more relevant for folks on the cutting edge, I think the latter is what a lot of the larger enterprises are thinking about, and as a result were what much of the audience was thinking about.

It’s worth taking a step back and thinking about how AI is landing in different orgs. At tech companies and startups, folks primarily interact with AI as a developer tool. Cursor and, later, Claude Code are interfaces that a dev spends all day in. The average developer at these companies is, as a result fairly intimately aware of what AI can do, simply because they interact with it so much.

That is not how AI has landed in other industries like banks and retail. In organizations where engineering is not obviously a critical function — places where engineering is more a cost center than where the money gets made — the primary way AI gets used is as infrastructure. The most common shape is the ETL pipeline, a recurring process that Extracts data from some table, Transforms it somehow, and then Loads it into another table. These sorts of tasks are a dime a dozen at any large enterprise, and less tech-forward companies will pay a lot of money to external consultants to build these out. Traditionally, ETL pipelines are brittle and required a lot of fiddling with APIs and different fields and so on. LLMs, with their inherent flexibility, promised to massively simplify that effort.

New technology never gets adopted wholesale, especially among larger slow moving companies with a lot to lose. Instead, new technology comes in as a supplement or a replacement for something that already exists. That’s a large part of why AI landed in ETL world for so many companies — because that was the best fitting preexisting shape to slot it into. Unfortunately, if you’ve ever worked with ETLs, you’ll know that you can’t just, like, screw around with the data. These pipelines need to be consistent and auditable, and an AI agent is neither.

Which brings us back to evals.

In large tech companies, tokenmaxxing meant ‘give individual developers unlimited budgets for things like Claude Code’. Everywhere else, tokenmaxxing meant ‘let’s throw more tokens at these failing ETL pipelines to get them from 80% correct to 100% correct’. How do you do that with AI? Well, one naive way is to build another AI system that checks the output of the first. But what if that one fails? Well, you could build a third system that checks the output of the second.

You see the problem.

Now that tokenmaxxing is dead, some of these less tech enabled firms are limiting employees to ~$50-$100 token spend per month. At full unsubsidized price, that’s barely anything at all, but then again most of these teams haven’t exactly been using Claude Code or Codex anyway.

I mention all of this mostly because I find it interesting to see the gap between SF and everyone else. I’ve said in the past that SF is in a bubble, and that was very apparent during AIE. The evals track talks were all about researchers and labs trying to figure out better RL environments, but a lot of the people in the audience were like “that’s great but tell me how to make my ETLs not be bad.”

As an aside, I think there isn’t really a good way to make these things consistent / auditable, nor should you really try. The long tail is too long. Instead, have a human in the loop. AI that goes 95% of the way and has a human approve or not has, in my experience, been way more successful than any AI system that attempts to go 100% of the way.

The folks who are really on the cutting edge spent a lot of time talking about background agents / cloud agents, and the ten-million reasons why people should use them.

Now, I recognize that as the founder/ceo of a company that is selling a background agent stack, I am a bit biased here, but it seriously felt like a full half of the talks at AIE were shilling for our product. There were talks about why you need background agents for productivity, and talks about why you need background agents for security, and talks about why you need background agents for compliance. There were even talks about why you need background agents for AI enablement! I’d say a solid half of the security track, all of the sandbox track, and many many other talks (including keynotes) were about moving things from local machines to the cloud.

(This post is brought to you by Nori! When you think “background agent” or “cloud agent,” think Nori!)

If you are unfamiliar, the name basically tells you whatever you need to know: background agents / cloud agents are agents like Claude Code that run in the cloud, in the background.

Why might you want this? Well, when Claude Code is running locally on your computer, it is:

• Stuck on your machine. If you close your laptop or walk away from it you cannot (easily) take the conversation with you on the go.

• A massive security hole. Everything on your computer is fair game. There is a fundamental tension between usefulness and risk, and unfortunately besides a few very very stressed out CISOs people are not thinking about this trade off at all.

• Unable to easily coordinate and collaborate. If you want to try and run multiple agents in the same code repository, you have to use tooling like git worktrees. If you run too many agents at once, they will happily eat all of your computer memory and kill the computer.

All of these problems are because the coding agents live on your computer, so…

Cloud agents neatly solve all three of these issues and a whole host of other things. The openclaw folks already showed how being able to access agents from anywhere was massively useful. Because they are centralized, you get massive leverage on security, observability, governance, and all those other lovely enterprise buzzwords. And cloud agents are a necessary step towards the ‘fully automated business’, because, by virtue of being not-on-your-computer, they can respond to events and timers and do things automatically.

It turns out this is an extremely powerful primitive to have. It’s a universal automator — you can literally fire off any event to this thing and have Claude respond, and Claude is a PhD level genius. We use background agents for everything from in-slack coding to company brain management to automatically responding to bug reports to onboarding new users.

These are still relatively new, but in ~6 months I think the market for these cloud runtimes is going to be really really hot.

The other big conversation topic among the teams on the cutting edge was about loops, software factories, and lights-off codebases.3 Or, well, that was ostensibly what people were talking about, but what they were actually talking about was something closer to “do we still need humans to do this stuff or are we all screwed?”

In some sense, a “loop” is a hack. LLMs suffer from context rot — the longer you talk to an LLM, the dumber it gets, until you hit a hard cap where you just can’t do anything with it anymore. It is a massive limitation, and a ton of “cutting edge” AI infrastructure is about solving this problem.

The loop is a very simple idea: what if you gave an LLM a big task and a scratchpad, and instead of having it try and finish the task in one shot, it takes on a small piece of the task and only does that small piece, and then writes out what it did. Then the thing loops to do it again. And again and again until it’s done.

while :; do cat PROMPT.md | claude-code ; done

People have been talking about loops for a while. The “Ralph Wiggum loop” has been in the water supply since last year. Karpathy’s auto research work is slightly newer, AlphaEvolve is slightly older, but they’re all basically about the same thing, which is “can you exploit the fact that LLMs never tire to turn every problem into a brute force search problem?”

I’ve written about this before, back in October:

I started using agents effectively when I began to see code as a search problem. Agents are like our chess players above. At each turn in a conversation or agentic loop, they make a decision. That decision takes them to a new state in the search tree, and the process repeats. Agents can go any which way down the search tree. Sometimes they will start on the right path and then veer off track. Other times, they will be wrong from the start. If you are using a coding agent, your job is to help it narrow down the search tree.

…

As a software engineer using a coding agent, your goal is not to get the best single shot code output. Your goal is to simply get the best code, period. If coding is a search problem, a very easy way to improve your code is by running many searches at once. So do that!

This raises some interesting questions. Is…software engineering a brute force search problem? What about, say, running a company? Can you be like “fable make me a billion dollars” in a loop and just sit back drinking mimosas in Fiji until it succeeds?

Several startups have staked their business on the answer being yes. And Matt Levine has joked that this is obviously the right business model for the big labs:

It really is the greatest business plan in the history of capitalism: “We will create God and then ask it for money.” Perfect in its simplicity.

But back at AIE, the premier AI conference where everyone was basically obligated to be AI-pilled, the general consensus was “eh, not yet.”

The agents are good, and Fable is really good, but a 3 month old codebase is just a totally different beast than a 1 year old codebase, and both of those feel really really different when people are actually out there using your tooling and you have random backwards compatibility requirements that you have to remember (or you go to jail, or at least lose a big contract), o, and yea, memory is something that the agents are uniquely really terrible at. Most of the folks who were defending ‘loopsmaxxing’ were doing so somewhat aspirationally, acknowledging that the loop-based-tooling wasn’t all there right now but would get there eventually.

For what it’s worth, at Nori we are running a “hybrid” software factory codebase. The entire codebase is agent written. I am definitely not reading most of the PRs that are being shipped. But also, I have a pretty good mental model of how the code is structured. We have regular daily tasks that automatically look for tests to clean up or abstractions to define (or destroy), like little cleaner fish that are just swimming around looking for things to fix up. And roughly once a month, we all end up doing a big chunky refactor of some part of the codebase that has gotten too hairy, often because we’ll eyeball a PR and go ‘wait why is <small feature> so many LoC changed?’ and then dig in with an agent to figure out if there is an abstraction missing somewhere.

Isn’t that kinda just how normal swe works, but on steroids? You build stuff, and you build more stuff, and eventually you look at the stuff you built and refactor a few things to make it easier to build more stuff on top?

Some people argue that when using agents, no one really has any deep model of the codebase, the kind where you would know a priori what files and functions need to be changed to make a new feature work without regressions. Which is true.

But I think that is true of basically every codebase, modulo time and size. How much of Google’s massive codebase did anyone really have in their head? Jeff Dean is superhuman and even he probably only knows like 0.01% of that thing. Which is to say, it’s worth investing in things that make the codebase legible — it will probably make the models better too. And beyond that, any maximalist position of ‘only use agents never look at code’ to ‘never use agents only look at code’ is wrong.

I think the loops will get better. A good friend of mine at one of the companies that had early access to GPT 5.6 Sol was telling me about these crazy subagent harnesses, where Sol was coordinating a bunch of Opus agents which were coordinating a bunch of GLM agents. Seems really impressive. But it’s also worth mentioning that he was overall unimpressed by the models’ collective ability to do things without a human.

This all sorta brings me back to the evals thread from earlier. I think the right architecture is ‘automated artifact, human review’. Our definition of what an ‘artifact’ is will continue changing as the models get better, but you need that human in the loop somewhere, either for the initial prompt or the post hoc analysis.

A few other miscellaneous thoughts.

• As usual, there was a lot of chatter about agents running other agents and agent swarms and things like this. Reading the tea leaves a bit, the most chaotic versions of this approach (“All the patrol agents Refinery, Witness, Deacon, Polecats — create wisp molecules for every single patrol or workflow run“) have somewhat failed to pass muster. I think the agent swarm thing was always somewhat linked to the loopsmaxxing tokenmaxxing software factory thing. The biggest problem with software factories is that agents are not very good at high level decision making, things like “what should I prioritize” and “what are the right trade-offs to make” and “what is the right way to break up a complex task into smaller interlocking useful pieces.” The agent swarm promised to solve these problems. I think the theory was something like “agents can handle low level decision making, so maybe all they need to handle high level decision making is a bunch of grunts to do the low level stuff and a metric ton of tokens.” Unfortunately, this framing is obviously flipping causality. I think a lot of the agent swarm hype has become more about orchestrating targeted subagents for things like research or code review, something I am / was already much more bullish about.

• A lot of people waking up to the security risks of having these little imps running around on every computer in a company. Half the talks were about putting the fear of God into various CISOs.

  

  

• “Multiplayer” was a surprisingly big topic of conversation, driven in part by Anthropic pushing their new Claude Tag slack bot in ~all of their talks. The basic idea is that when you have a lot of people collaborating with an agent at the same time, eg through slack, the experience of writing software — or even running a company — becomes fundamentally more collaborative. As someone who is selling a no lock-in version of Claude Tag, yea, I basically agree. Having a really good integrated AI setup is transformative, in part because it meaningfully changes everyone’s relationship to engineering. These tools actually remind me a lot of Figma. When Figma came out, it was obviously a great wireframing tool, but the real magic of the system was how it allowed everyone to participate in the design process because it was multiplayer by default. Even though the form factor is totally different, tools like Nori or Claude Tag have the same effect on engineering as Figma did on design. Suddenly, everyone can participate in the engineering process, no arcane runes required. This is, again, another pitch for background agents generally.

• One thing I disagreed with most strongly was the idea that everyone in an org should figure out how to use AI ‘properly’. I get the impulse. As AI adoption sky rockets, knowing how to move around an LLM will increasingly just be part of the job. But there is so much behind getting an AI agent to “work” in a business setting that I have a hard time imagining that some day job 9-5 guy is really going to spend the time figuring out how to do it right. It doesn’t make sense to have, like, your sales guy figure out how to properly do security for agents running on his laptop. He’s just going to do whatever works as fast as possible to get back on the phone selling, which means it’s up to the rest of the company to make the fastest path the happy path. To steal a quote from Dex, it doesn’t make sense to have everyone in your org figure out how to use Kubernetes even if it’s really important, why would it make sense to have everyone in your org figure out how to run agents properly?

• There was a whole bunch of stuff on open source LLMs (GLM 5.2 was a particularly large topic of conversation), auto research, robotics, and model training that I unfortunately missed. Lots of talks, maybe too many. I was hoping to see more researchers but it was very much more an industry conference than an academic one.

Some of my favorite parts of the conference were outside the conference. There were a bunch of sponsored side events, including one great one from Modal (thanks Monishee!), but I’m mostly talking about catching up with other folks who happened to read the blog and invited me to come hang. Office Hours with the LLM whisperers was very fun (thanks for having me ), so was dinner with the botco crew while watching the world cup. On the flip side, things were so busy that I didn’t get a chance to really see my usual Bay area crowd — if I missed you this trip, big apologies, and I will see you next time. For everyone else I met, hit me up if you’re ever in NYC!

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. We organize in person meetups on our luma calendar. More articles about AI here.

If you are trying to build background agents, we can help! We build white-labeled custom background agent infra that work like Ramp Inspect or Claude Tag, with no lock in at any part of the stack. Check out noriagentic.com for more.

Agents are increasingly picking up more and more of our workload. We use agents to write and ship code, publish our dev blog, and even write our board decks. To make that all work, we have to give our agents access to more data sources. Slack, email, GitHub, aws, linear, datadog, sentry. For most of these tools, we can just pull an MCP server and call it a day. But MCP is extremely token hungry, and often implemented really poorly. For some critical integrations, we want to own the integration point.

When building out a new integration, we often reach for CLIs over trying to build a custom MCP. CLIs have built-in progressive disclosure (they do not suck up tokens with parameter descriptions for function calls unless your model absolutely needs to know). The models are extremely familiar with using and calling CLIs through bash. And the models are very good at writing efficient CLI tools, because they have tons and tons of examples.

CLIs that are built for agents as their primary user behave differently than those built for humans. We’ve spent a lot of time thinking about what that behavior difference ought to look like. In no particular order, here are a few principles we use when building CLI tools that assume agents are the primary consumer.

• No colored output. No spinners. No animations. These waste tokens and are probably unappreciated by an agent that can’t see the visual media.

• No interactivity at any point. Everything must be accessible through flags and stdout/stderr output.

• Calling the cli with empty, --help, --list, etc. should list a full description of what the tool can do at a high level and descriptions of individual commands (”use this command for…”). Overlap is fine here, we want to make it easy for the agent to stumble onto instructions.

• Help messages should be recursive. Calling foo --help should explain what the CLI does broadly. Calling foo bar --help should explain what the bar command does within foo, including any parameters that bar may need. Aim for 100% docs coverage.

• Every error or help message should include a file path for where the relevant code path is stored on disk, and a message encouraging the model to look up the source if needed.

• Related: we generally install the CLI by building from source or bundling source where possible, so that the decompressed source is present for the model to figure out what to do.

• Make it easy for the agent to fix itself. If the agent tries a command that doesn’t work, do a nearest neighbor lookup. “Did you mean…” but for CLI commands.

You can find all of this in skill form here. Just drop it into your agent of choice, or run

npx nori-skillsets install amol

to download my personal agent config (which includes the agentic cli skill already in place).

We’ve built a few different CLI tools that follow these principles, including:

• nori-slack-cli, a thin command-line wrapper around the Slack Web API that maps 1:1 to Bolt (@slack/web-api). Every method Bolt exposes is reachable through a single dynamic command — there is no curated subset, no opinionated abstraction layer, and no business logic. If Bolt can call it, this CLI can call it.

• nori-newsletter-cli, a CLI for managing and sending newsletters via AWS SES. Handles subscriber management through SES contact lists, CSV import, and individual email sending with automatic unsubscribe support.1

• nori-luma-cli, a wrapper around the Luma event manager API, so the agents can plan your parties for you! 🥳🥳🥳

• nori-discord-cli, which is like the slack cli but for discord.

• nori-shopify-cli, which lets your agent access the admin tools in shopify.

All of these are open source, so feel free to drop them with your agents and see how they use the tools.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are trying to build background agents, we can help! We build white-labeled custom background agents that work like Ramp Inspect for our customers. Check out norisessions.com for more.

Guest post from Clifford, my CTO and co-founder at Nori. Original here.

For the past year the techfluencers and executives and tokenmaxxers have been telling us that code is cheap. Lately harness engineers have upgraded the claim: code is free.1 Soon enough, the marginal cost of acquiring another thousand lines really will round to zero, totally untethered from the old bottlenecks of human time and human expertise.

I think this misses the point. Sometimes free isn’t cheap enough.

Over the last century, conservation efforts for American bison in places like Montana and Nebraska worked a little too well. The buffalo came back — and then they kept coming. Herds grew to the point where they threatened serious destruction of the very ecosystems we’d set out to protect. So every autumn, the surplus animals had to be culled. From the 1960s up until around 1992 the Department of the Interior ran a surplus wildlife disposal program to try and find an outcome other than slaughter for these beautiful animals.

Which leads me to ask you, the reader: how cheap would one of these buffalo have to be for you to sign up?

If you answered “I wouldn’t, at any cost” — then I hope you see my point. I bring up this specific example because for decades, these buffalo were actually available for free!2

Turns out there can be costs to a transaction other than just the acquisition cost. Some are financial. Some are opportunity costs. Some are just risks and inconveniences. A buffalo costs an enormous amount of time and money to maintain. Every hour you spend caring for it is an hour you didn’t spend on the work you actually meant to do. And a new buffalo can expose the rest of your herd to new pathogens and disease.

Those other costs can matter so much more than the sticker price, that the up-front number becomes basically irrelevant to the decision. Free was never the question. So if over the next couple of years the frontier labs decide to debt-finance a massive supply of buffalo and hand them out for nothing... I would not recommend rebuilding your entire way of working around buffalomaxxing.3

Because sometimes free is not cheap enough.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are trying to build background agents, we can help! We build white-labeled custom background agents that work like Ramp Inspect for our customers. Check out norisessions.com for more.

We have found zero performance benefit on SWE tasks when agents have search access to their previous transcript sessions, provided they have access to other forms of context. We also have not found much benefit in trying to automatically trawl through session transcripts to improve agent context, unless there is a human in the loop.

This was pretty surprising.

Intuitively it feels like there's a lot of valuable information in a transcript between an agent and an engineer. Maybe it would have information about why the code exists, about user intent. Or it might have the other approaches that a user tried and discarded. At the least, it would have some amount of additional context that the agent could use to augment its understanding. I believed this so strongly that my company built an entire product around this concept. I used to tell folks that "session transcripts were the new oil," that they were more valuable than the code itself.

Other people have clearly had similar thoughts, which is why there are so many different tools to do session backed memory, including (of course) Claude Code itself.

I think the most common architecture is to do something like:

• Store all transcripts across an organization in a DB

• Put a vector search, an elastic search, or a SQL search layer in front of it. Ambitious teams will use all three. Maybe graphs will be involved.

• Make this available to the agent using an MCP, or by exposing a cli with skills.

For us, this additional work doesn't seem to make a bit of difference. If anything, based on many months of testing with and without session search access, it may make the models worse.

Why might this be true?

One thing our team cares a lot about is coding artifacts. We don't really write code by hand anymore. In order to make PRs legible, we emphasize good commit messages, good pr messages, and comprehensive documentation. Every code change comes with extensive metadata that is committed alongside the code. When our agents do work on a piece of code, they are instructed to go look at the docs and the previous PRs.

In other words, the agent is already distilling all of the information that is valuable about a transcript, and storing it where it is needed and easily accessible. So when the agent uses a transcript search server, it ends up spending tokens reading things it already knows, while picking up all the stuff that the agent decided not to write down in the first place. Maybe, every now and then, there's some useful nugget of information in there. But most of the time, the agent is just looking at a pseudo nonsensical scratch pad and wasting precious tokens to do so.

The agents are also terrible at actually removing context, which is a critical capability for maintaining long term memory. I mean, across literal thousands of sessions, I've never seen it happen even once.

This is not a trait that can be removed with some clever prompt engineering. Agents don't have state, so they have to assume everything in their input context window is the ground truth. Every line of code, every existing bit of memory, every token is treated as an expression of intent -- even if that code or that memory was generated from a random decision made by some previous agent session, never reviewed or even understood by a human. This intent drift compounds the more the agent tries to autonomously build up a memory base.

As far as I am aware, there are zero coding benchmarks that assume the input data is corrupt. In fact, the models are penalized for assuming that the input data is wrong. This is partially an alignment issue too -- we don't want to have agents doing unintended things, and there isn't an easy way to thread the needle of “don't delete the codebase” and “do delete some of the input context.”

Since models can't actually garden their own memory, automatic memorization ends up in the same place: a load of garbage eating tokens, bloating bills, and degrading model quality.

Net net, I've become really bearish on tools that index and store and surface in session transcripts to an agent. The session transcript may be useful for team observability, but it won't make your agents better.

That doesn't mean that agents have no role in learning context over time. We use our internal nori bots to review everything that happened at the company each week across PRs, slack, drive, etc. And they then propose a set of changes to our built in nori skillsets, tagging the team in slack. These are all default rejected. In order to accept a change, you have to go in and actually look at the diff and make sure it fits the intent.

We accept less than 20% of these. Which means 80% of these “automatic” updates would've made the model worse. I can't imagine how much more unsustainable that would be if a multi-hundred person org were all saving these “updates” automatically all the time.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are looking to build background agents, we can help! Learn more about Nori at noriagentic.com, and our background agents product at norisessions.com.

Generally speaking, if you spend tens of thousands of dollars on something, you want to see something come out on the other end. Some return on investment.

O sure, not always. I’ve previously said that selling to consumers is sorta funny because they love spending money on things that waste time or actively cause pain. This is part of why the gambling apps are so popular these days. Why yes, I’d love to spend $100 on betting that Wemby scores a 3 pointer while doing a handstand and singing the national anthem in French.1

But for businesses? I’ve basically never heard a business leader say that they were going to set a bunch of money on fire because it made them feel good, at least not the same way a whale will spend thousands on Genshin Impact gatcha pulls. Like, imagine if some serious business leader, like, idk, Mark Zuckerberg, decided to announce that Meta was going to burn money. He could do that. He’s got the voting shares. But it would be a bit silly, wouldn’t it? I generally think if you’ve gotten to the point where you’re running really big really important companies, you mostly aren’t doing things for kicks, with one big exception.

If you haven’t heard, tokenmaxxing is (was?) a phenomenon where executives accidentally encouraged their employees to burn a bunch of tokens on useless tasks. The canonical example of this is, by complete coincidence, Meta, which has been thoroughly skewered for tying performance evaluations to the amount of token usage per person. Obviously, obviously this was going to lead to people just burning tokens on nothing. One of my friends at Meta reported that they literally would just have two agents talking to each other throughout the day to get her token numbers up.

This was such an obvious outcome that many people rounded this off as “these business leaders are really dumb because they decided to burn a bunch of money on tokens without expecting any return.”

I understand why that’s a tempting take, because that is kinda sorta what the public face of a lot of this was. But I’m going to do my favorite thing in the world, which is be a bit contrarian. It wasn’t that “executives accidentally encouraged their employees to burn a bunch of tokens on useless tasks.” Rather, “executives purposely encouraged their employees to burn a bunch of tokens on useless tasks.”

I work with a lot of teams on figuring out how to use AI effectively. A few months ago, there were a lot of people who were extremely resistant to using AI tools at all. Senior people, people that had a lot of respect in the organization. It was very difficult to convince these folks to use the tools. And when you did, they would often accidentally (or purposely?) use the tools in a way that would obviously lead to weird or bad outcomes.2

One way to think about the top down tokenmaxxing policies is that this was a technique by executives to break through. Yes, it was obviously a blunt force policy, but sometimes you need blunt force to break through a wall.

Of course, that was the situation a few months ago, when there were still holdouts. It’s now a few months later, and the tokenmaxxing policies had their intended outcome: everyone is using AI to code, at least a little bit. Most teams haven’t yet figured out how to build their own Ramp Inspect or Stripe Minions (if that’s you, reach out — we can help!) but basically everyone is at least using cursor in the side bar. Which, of course, means that token spend has gone way up. Unfortunately, but probably not unexpectedly, the increase in token spend has lined up with both OpenAI and Anthropic trying to go public. Both companies have limited the amount of juice their subscriptions provide while jacking up their API pricing. Token subsidies are increasingly vanishing.

So now the incentives are mostly gone and the cost is way up and, of course, teams are starting to roll back their unlimited-token-spend policies. All of this to say, tokenmaxxing is dead.

Except…maybe not.

The promise of AI tools generally is that you can have them run without human supervision to accomplish really hard and really tedious tasks that still need to be done. The big code migration, doing research on all your competitors every morning, keeping up with the stream of inbound and outbound — these are all things that people mostly hate doing and want AI to do.

Up until recently, though, you couldn’t reliably have an AI run for long periods of time. If you tried, you would notice that small errors introduced by the models (including hallucinations) would take on a life of their own and eventually become irreversibly embedded into the project. In the business we called this “compounding error.” It not only required a fair bit of human supervision, it also kept token costs low because there was little benefit in running agents 24/7 to begin with. Like, what’s the point of running a little demon in your computer over night if the thing is just going to tear up all your hard work? If spending more tokens results in worse work, you obviously aren’t going to spend more tokens!

That’s no longer true. We’ve entered a different regime, where spending more tokens generally results in better results. We call this “compounding correctness” — the more tokens you spend on getting a task correct, the more likely you’ll get a good outcome. We talked about this a bit at the last in person Agentics meetup:

Compounding correctness flips the calculus. If more token spend leads to better outcomes, then you’re going to want to spend a lot of time running tokens. Which sure as hell sounds like tokenmaxxing to me! The original incentives to tokenmax are gone, but eventually folks will realize that a new and more powerful incentive has take its place.

We’ve already seen some of this take place in the cyber security world:

Last week we learned about Anthropic’s Mythos, a new LLM so “strikingly capable at computer security tasks” that Anthropic didn’t release it publicly. Instead, only critical software makers have been granted access, providing them time to harden their systems.

…

This chart suggests an interesting security economy: to harden a system we need to spend more tokens discovering exploits than attackers spend exploiting them.

AISI budgeted 100M tokens for each attempt. That’s $12,500 per Mythos attempt, $125k for all ten runs. Worryingly, none of the models given a 100M budget showed signs of diminishing returns. “Models continue making progress with increased token budgets across the token budgets tested,” AISI notes.

If Mythos continues to find exploits so long as you keep throwing money at it, security is reduced to a brutally simple equation: to harden a system you need to spend more tokens discovering exploits than attackers will spend exploiting them.

You don’t get points for being clever. You win by paying more. It is a system that echoes cryptocurrency’s proof of work system, where success is tied to raw computational work. It’s a low temperature lottery: buy the tokens, maybe you find an exploit. Hopefully you keep trying longer than your attackers.

Fable is, tragically, gone now. But the underlying concept here still remains.

This is also in part why people are suddenly so excited about ‘loops.’ Boris Cherny, the creator of Claude Code, got up on stage and said ‘loops’ and everyone freaked out. The basic idea behind loops is that you run an agent until it reaches the end of its turn, and then when it finishes you simply restart the same prompt. With a bit of cleverness you can take a pretty heavy specification and automatically have the agent split it into parts and solve it over time. No human supervision required.

Is this some new thing? No, not really. The loop concept has been around since literally last July. It used to be called a “Ralph Wiggum loop,” but as the industry has matured so has our sense of humor and the ‘Ralph Wiggum’ part was dropped.

There were ways to get loops to work, but it was hard. You had to think a lot about how to prompt the agent, which in turn required a pretty deep familiarity with how these things work. Now, though, it’s easy. Compounding correctness makes it easy. You can basically prompt the LLM however you want and to a first approximation, it will do better every iteration of the loop.

So is tokenmaxxing really dead? Maybe temporarily, but long term I don’t think so. Teams that are at the cutting edge are currently building or have built the infrastructure necessary to run agents 24/7. It’s only a matter of time before the bigcos realize that the cost benefit has shifted again.

The real winners here are the open model platforms. Tokenmaxxing the top labs will never stand up to any amount of CFO scrutiny. As open models get better, it will become more popular to simply run those in a loop. That was the core thesis of Rohan’s talk above. If Claude gives you 1.1x improvement per iteration, and GLM 5.2 gives you 1.05 improvement per iteration but costs ~5x less, you can just run the second loop 5x more times and it will be better.

The last thing I want to mention here is that some of the ridiculous token spend is downstream of a serious misunderstanding of the best way to use these tools. Before coding agents really took off (thanks in large part to much better harnesses like Claude Code), lots of people were making their own custom agents. And that was legitimate work! You had to think about this stuff as if it was…well, software. There was an art to figuring out the tools and the prompts but the core of it was still just software, even if it was supported by ‘AI native’ frameworks like Pydantic or Langchain.

You can’t fit a square peg into a round hole. Executives across the board saw this style of building agents, went “o, this is just a more flexible zapier workflow,” and proceeded to demand data processing pipelines that could do one-off tasks that were ‘agentic’ instead of building those same pipelines in good ol’ deterministic code. ‘I need to do data labeling, so I will build a data labeling agent’, that sort of thing.

Now, relying on an agent to do some of this stuff is already going to be significantly more expensive than just doing a workflow automation. But the bigger issue is the accuracy: none of these ‘agents’ ever really took off, because they were never as correct as a deterministic pipeline would be.

If you’re committed to using agents but want to reduce the cost of hallucinations and things, what do you do? Why, you build another agent! A ‘quality checking’ agent, or something like that. And what if that agent gives you errors? Well you’ll just build another! And now you have 3x the token cost, enjoy!

The story of tokenmaxxing is, again, one of RoI. That story didn’t just play out at the bigtechcos. It also happened at a less advanced scale at companies all over the country — companies who poured billions into random agent pipelines built by one off consultants that unfortunately never really quite worked all that well.

Notice that these are actually two different kinds of tokenmaxxing.

• The first kind is ‘spend a lot of money on tokens for your developers‘. Here, devs are using tools like Claude Code and figuring out how to run things in loops and using a lot of tokens to do it. Ostensibly this is a good use of money because it’s making the engineers themselves more productive.

• The second is ‘spend a lot of money on tokens for your pipelines‘. Here, devs are still writing code by hand! They are using that code to create one-off agents to do very specific tasks often in a non-deterministic and brittle way, and it’s those agents that guzzle up all the tokens. This is only a good use of money if the pipelines work, which they don’t.

But here, too, we are seeing a shift. Increasingly, these sorts of one-off pipeline-based tools are better done by generalist platforms that are skinned for the specific task, than an “agent” specially designed to do that one task. There’s some market arbitrage here. Some buyers haven’t realized that generalist agents have gotten really good, so they will go to consultants asking to ‘build me an agent’, and the consultant essentially writes a skill file and says “that will be $2m please.”

Luckily, this too shall pass. Generalist model platforms are obviously the future for anyone who has used them (and if you haven’t, again, reach out!) And that, again, will lead to another rise in tokenmaxxing behavior in this part of the market.

The natural end state of all of this is the ‘software factory’ or, even further, the ‘dark factory’ — a codebase that pumps out code, reviews code, fixes bugs, writes tests, and so on without any human supervision. The human simply puts in a spec and out comes an application. The folks over at StrongDM have taken this to the furthest extreme, arguing that engineers should aim to spend $1000 in tokens per day. This is almost certainly hype, part of a long trend of saying egregious things to get coverage and buzz. We have a software factory, and we spend like $600 per month. But the hype and buzz comes about because, even though it is currently ridiculous to spend the price of a senior google engineer in tokens per engineer, there is a kernel of truth to this. The incentive to spend ludicrous amounts of money on tokens are there, latent, waiting for diffusion.

What’s old is new again and what’s dead may never die. Tokenmaxxing is dead, but we haven’t seen the last of tokenmaxxing just yet.

Other things:

• GPT 5.6 is out, kinda sorta. From the announcement:

  We’re beginning a limited preview of the GPT‑5.6 series: Sol, our flagship model; Terra, a balanced model for everyday work; and Luna, a fast and affordable model. Terra has competitive performance to GPT‑5.5 while being 2x cheaper and Luna brings strong capability at our lowest cost.

  …

  We believe in broad access, and we plan to make GPT‑5.6 Sol, Terra, and Luna generally available in the coming weeks. As part of our ongoing engagement with the U.S. government, we previewed our plans and the models’ capabilities ahead of today’s launch. At their request, we are starting with a limited preview for a small group of trusted partners whose participation has been shared with the government, before releasing more broadly.

  …

  We don’t believe this kind of government access process should become the long-term default. It keeps the best tools from users, developers, enterprises, cyber defenders, and global partners who need them. We are taking this short-term step because we believe it is the strongest path to broader availability in the coming weeks, while we work with the Administration to develop the cyber Executive Order framework and a repeatable process for future model releases.

  Washington Post was even more aggressive in its analysis:

  U.S. government will decide who gets to use latest upgrade to ChatGPT

  The Trump administration came to power preaching a laissez-faire approach to AI but has lately increased oversight of the industry.

  Reading between the lines, it looks like we are getting more government regulation of AI companies, but unfortunately the process as it currently stands is completely opaque. Reactions are mixed. On the one hand it’s great that the administration’s previous attacks on Anthropic are also being applied to their competition. On the other hand, it seems like the process is totally opaque, and now the government is unilaterally picking winners and losers not just among AI companies, but among every other industry where AI may matter (i.e. all of them). The companies who get to use OpenAI’s tooling (and Mythos, see below) are currently unknown. It would be very concerning / disappointing if we find out that it is exclusively companies with ties to this administration.

• Related: Mythos is back on the table, at least sort of. From Semafor:

  US releases powerful Anthropic model Mythos to some US companies

  The US government Friday lifted its block on Anthropic’s powerful Claude Mythos 5 AI model, allowing the company to release it to more than 100 US institutions, including major companies and government agencies.

  The decision, in a letter sent Friday afternoon to Anthropic, is a major de-escalation in the confrontation between the Trump Administration and one of the world’s most valuable private companies. Two weeks ago the administration imposed export controls on Mythos, leading to a shut down of the model and its cousin Fable 5 after warnings from Amazon and other companies that they could be “jailbroken” for malicious purposes.

  The letter is silent on Fable 5, a weaker version of Mythos that was briefly the most powerful AI model widely available to consumers. People close to the talks said they are moving toward releasing Fable as well, though that timeline is unclear.

  “I have determined that appropriate safeguards are in place to permit certain trusted partners to access the Claude Mythos 5 Model,” Commerce Secretary Howard Lutnick wrote to Anthropic’s chief compute officer Tom Brown Friday, citing “significant progress” in the intense, daily talks between the government and the company since the block went into effect.

  Again, picking winners and losers.

• While we are talking about OpenAI’s launch, one thing that was buried in the announcement was that the tooling would be available on Cerebras’s high-speed inference machines at ~750 tokens per second. This is quite fast. Right now we are in a regime where it makes sense to treat AI tools as async operators that can go off and do tasks without supervision. But that is mostly downstream of the fact that AI is slow and it takes a while to do things. If AI was really really fast you would plausibly go back towards a more synchronous model of operating. For an idea of what this may look like, check out https://chatjimmy.ai/. It’s just a demo, but wow, what a demo.

• Open models like GLM 5.2 have gotten pretty damn good. They aren’t SotA, but they are much cheaper than their frontier equivalents. Right now, GLM 5.2 is ~$1.4 per million tokens and ~$4 per million output tokens. By contrast, the entire Opus 4.X series is $5 per million input and a whopping $25 per million output. The only model in the Anthropic suite that even comes close to GLM 5.2 in terms of pricing is Haiku 4.5, at $1 / million input and $5 / million output. But GLM 5.2 blows Haiku out of the water, and in some cases is even stronger than GPT 5.5 on benchmarks. If I were the big labs, I’d be pretty concerned about this. And if I were basically any consumer on the market, I would be doing everything I can to avoid provider lock in by adopting tools that are able to sit on top of all of the major players.

• OpenAI unveiled an in house chip for inference.

  On Wednesday, OpenAI unveiled its first custom-built inference processor, designed and manufactured in collaboration with Broadcom. Named Jalapeño, the new processor was designed specifically for the unique needs of OpenAI’s inference systems. OpenAI’s own AI models assisted in the development of the chip, the company said.

  Jalapeño, like a jalapeño chip. Get it?

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about how Nori can bring your company into the glorious AI future at norisessions.com.

The year is 2022. I am the CTO of my first company. I am managing a team of brilliant engineers and a team of brilliant and completely non-technical sales and marketing folks. I spend, to a first approximation, half my life counting geese.

I am constantly in meetings. Every meeting has some complicated name, like ‘Aligning strategy for Q1’. But I know that the meeting is really about counting geese. We look at dashboards that make sure that everyone has their geese properly counted. Sometimes the engineers will mention that there were geese that they counted that aren’t reflected on the bean counter. Other times, the marketing and sales folks will say that some geese that we thought had been counted hadn’t been counted enough. There is much argument about the true number of geese.

Once we reach consensus, we spend time making sure that the goose count on GitHub matches the goose count on Linear which matches the goose count in Google Drive. There are more geese, the flow of geese is constant, so we have to spend an increasing amount of time counting them. We plan years around geese. I try to protect my team from the geese. Mostly that means that I am spending more of my time counting geese.

This is at a 14 person startup. It was worse when I was at the big tech co.

The year is 2026. I’m CEO of my second company. I haven’t thought about geese once in the last 6 months. The AI counts the geese now.

There are a lot of things to be worried about with AI. Things that I’ve written about extensively in other parts of this blog. But the thing that I am most excited about is the complete and total elimination of bullshit overhead. The future of work may be complex and crazy. But you know what it won’t involve? Having a founder staying up late at night making sure the linear story points are aligned with the things merged into GitHub so that the eng team sync meeting can save fifteen minutes of everyone’s time.

Everything that people hate, they hate because those tasks are boring and routine and dull and mindless. How many teams love to hate their CRM? Every time I get off the high of a fantastic customer pitch, I have 0 desire to then go into Hubspot or Salesforce to mark some row in a table ‘Lead’. How many hours get lost to making slide decks the right branding? Raise your hand if you lost an hour trying to figure out how to move an image that is in the background of your slide because it’s set as a ‘company wide template’, which was a feature you only just learned about ten minutes ago. How much information gets buried in slack, or docs, or linear, or wherever you do your work? Hours and hours of productive time wasted counting geese.

AI tools excel at this kind of data munging. They are so good at taking in data from calendly and piping it into salesforce or pulling in sentry data to debug something in github or whatever. Every company has a “brain.” Normally that brain is decentralized. It exists across the ten million different tools we all use to track state. Jira, confluence, notion, GitHub, linear, Asana, Monday, trello, docs, drive, slack, teams, one drive, Dropbox, one note, Salesforce, intercom, otter, fireflies, snowflake, databricks, datadog otel AWS gcloud Coca-Cola double decker bus his name was my name too.

The promise of AI is that you never need to think about any of that again. A universal AI operating system, a single entry point to the entire company brain. That doesn’t mean replacing people, though some folks may see it that way. To me it means taking the bullshit out of bullshit jobs.

I don’t think everyone is going to get there all at once, but the shape of the future is very clear to me.

• There are humans and there are agents.

• The humans spend time making decisions that matter — north star vision, strategy, marketing, and actually talking to people.

• The agents live in the background, in the ether, ephemeral, automatically bringing the vision to life with minimal friction.

This is a new and transformative way of conceptualizing work. AI is a new and transformative technology.

But it’s also very much within reach. There are teams who are “full auto” already, who are producing what would be Herculean amounts of output with a single person.

The main thing you need to get from point A to point B is a bit of forward thinking and some cloud infrastructure. We already provide the latter. Let me show you a bit of the art of the possible, in the hopes that you’ll get some of the former.

Here’s my morning a few days ago.

I wake up around 8:30AM. I walk into the kitchen and grab my phone, which I purposely try to keep in another room to avoid doomscrolling at night. There are like five slack messages waiting for me. I had a fleet of agents that were working through the night. I quickly eyeball the results while brushing my teeth.

One of them spun up at 3am to fix a bug that was reported by sentry. It posted a full root cause of the bug and a PR to fix it. I glance at the changed line diff and the pr description. Looks about right for what I’d expect for this, I happily merge it without looking at the code. I used to look at code, because I was worried about slop. But our agent hasn’t produced slop in 3 months. At some point you’ve got the thing so dialed in that it just works. I go to do some physical therapy stretches for my back — too much time coding in bed.

Shower, out the door, try to get on the Path train by 930AM.

While I’m sitting at the subway station I look through the other tasks. Two of them are confirmations to send out draft emails responding to some customer inbound. The agents personalized the responses, using a combination of call transcripts and website research. I tweak them both — I still can’t quite get the AI to nail my voice — but both emails go out in about 5 minutes. For someone who normally agonizes over each word in an email, this is a miracle.

One of my agents came back with some market research for the day. Things that the engineering hive mind on Twitter are thinking about. It is amazing for my mental health to be able to keep up with Twitter without being on Twitter. I chat with that agent a bit in slack, asking it to dig into a thread about agentjacking.

Another agent sends me a list of new papers on arxiv about representation learning. I spend the first half of the train ride reading through the summaries, asking questions about them and how they work at the stops where some 5g manages to leak down into the station.

Half the train is wearing Knicks gear, and that got me thinking that I should do something for our website. I open slack and type this prompt:

In between stops I iterate with the agent, all in slack. My cofounder sees what I’m doing and tells the bot to add a dunk animation in the same thread.

It’s done by the time I hit the 23rd stop and get out of the train. I can’t say that I have a ton of aesthetic sensibility, but I kinda like how it all came out.

By the time I get to my desk another agent has kicked up automatically, giving me an update on all of the changes in the code base yesterday alongside a brief summary of what everyone worked on. No painful daily sync, where you multiply every wasted minute by the number of people in the room. The rest of the team is locked in, talking to the agents from their CLI or slack or even lounging on the sofa on his phone. One of them is pacing around talking into his phone, recording a transcript to kick off a bunch of tasks on the go.

I am more productive on my way into work than on most full work days in 2022.

My team does 70% of our feature dev work in Slack. 30% of it from our phone. All of it through agents (we don’t write code and haven’t in months).

We do 90% of our ops work using automatic workflows that we never think about, or in slack through the agents.

We do all of our debugging through the agents through slack, though really it happens automatically before we even know there’s a bug in the first place.

We have the agents make slide decks. We have the agents update the website. We have the agents manage our CRM. We have the agents make sure linear and github are in sync.

My team gets to spend our time doing what we love. We think about product. We think about our customers. We write about things we care about, like this post. When people talk about AI being the future of work, they think that AI will get to do all the fun stuff. So far, for me, the AI has just automated away all of the things I hate doing.

And that’s really the goal, or at least should be for anyone thinking about how to deploy AI successfully.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about how Nori can bring your company into the glorious AI future at norisessions.com.

Guest post from Clifford, my CTO and co-founder at Nori. His posts always do way better than mine, and it’s a shame he doesn’t write as much as I do because he has real insights whereas I mostly just shout my opinions at the cloud. You can see the original here.

Also I will be in SF from the 29th to the 3rd! DM me if you want to catch up!

Also, also, if you or your team is trying to figure out how to use AI at your company — if you are struggling with costs, or with security concerns, or with roll-out to nontechnical staff — take a look at Nori Sessions! It’s now generally available. On average, Nori makes teams 2-5x more productive across sales, ops, and eng. AI promises the future of work. Nori Sessions is how you get there.

The Wrong Scoreboard

The discourse on coding agents has been obsessing for the past year over the wrong question. The main focus has been what models can do: lines written, autonomous minutes, benchmark scores, model cards, percent of lines shipped by AI. These are all generalized measures of implementation throughput. Useful for a bird’s-eye view of model progress, but they say almost nothing about where the actual bottlenecks now live. The operative question for practitioners in 2026 is not what tools can do, it’s what you should ask them to do.

Answering the “should” question requires a different lens than the capability benchmarks provide. Every task you might hand to a coding agent has two costs that matter: the cost to implement (C_i) — the time and expertise needed to produce the code — and the cost to verify (C_v) — the time and expertise needed to confirm the code is correct. The relationship between these two variables determines whether delegation is a net win or a liability.

Aside: About “Delegation”

When I first outlined this in November 2025, I was comparing handcoded vs AI-delegated implementations. My workflow has changed significantly since then: I rarely hand-write code.

The relevant choice for me is now between pair programming with the agent (high-touch, Socratic, every structural decision is guided) and delegating (agent leads research, planning, and implementation; you just review the output of each phase). The pair programming model is mentally just as involved as writing code, but mechanically faster. The delegation model is now very different, allowing you to run and ship five separate feature PRs in parallel (not some clickbait Xitter “I ran 100 agents in parallel today” make-work slop, but five actual product increments, in parallel, in a brownfield codebase).

Whatever the threshold of delegation is, in my experience the framework below applies.

The Two-Variable Framework

When both costs are low, it doesn’t matter what approach you take — the task is trivial either way. When C_i is high but C_v is low, delegate freely; the implementation is a job for the agent, and you can cheaply confirm the result. The inverse is equally clear: when C_i is low but C_v is high, build a detailed mental model by taking part in every step of the process.

The dangerous quadrant is top-right. When both costs are high, there’s a huge incentive to spin the slot machine many times, and see if the agent just happens to nail the task. Compared to hand coding, where you burn days or weeks to ascertain the quality, the agent might have a chance to succeed at the same or higher quality after just 60 minutes of work. For complex or off-distribution work, it may be a small chance... but that makes it even more tempting!

By skipping the mental effort, you go in blind on an equally demanding task: verification.

This is the trap.

The models have dramatically compressed C_i across the board. C_v has not moved at the same rate — and in many cases, without careful developer intervention, it has gotten worse.

Vibecoding and the Unaddressed Variable

Vibecoding is the logical extreme of treating C_i as the only variable.

Previously, architecture decisions were bottlenecked by implementation cost. Releasing that constraint completely, without addressing verification cost, is a big failure mode. Any frequent flyer on Claude Code has experienced this, as an end user of an entirely AI-coded application — the constant issues with UI bugs, unintended changes to history cells, broken permission models...

I've written about the flickering issues before, and I've been annoyed that sandboxing persistently pollutes the workspace with empty files (this issue has been recurring in different forms for three months now). Users of the Claude Web environment, of Cursor, and many other almost fully AI-coded products experience the exact same degradation of quality as the software grows so rapidly. It’s not just that more features lead to proportionately more bugs. When you don’t build a mental model of the codebase, you’ve skipped your first pass on verifying the logic, and you’ve gone without a map of what parts need verification. The consequence isn’t just bugs — it’s verification blindness: you don’t know what you don’t know.

This is a common failure mode that many teams have fallen into, particularly startups that feel the keenest urgency to ship faster.

Verification Debt

As a result, the most common form of tech debt in these highly agentic codebases comes from growing your feature surface area too fast and too loose. Every agentic feature shipped without a corresponding verification investment degrades your ability to autonomously ship *future* features. This is a compounding liability, not a fixed cost — first it accumulates, and then because these changes can have cross-cutting technical concerns, or act as bad examples for future work, it compounds. Unit and integration testing become slightly more important, to compensate. But E2E behavioral verification becomes far more important, because that’s the layer the agent generally cannot self-evaluate on its own. Skipping this investment creates verification debt.

Detour: What Spec-Driven Development Gets Right (and Wrong)

The popular framing of spec-driven development is wrong on two counts. It’s not about making prompt copy-paste easier, and it’s not about closing the loop on “Ralph Wiggum” workflows — generate, test, regenerate. These framings chase short-term speedups that don’t touch the real bottleneck.

The original insight of a specification is much more important: you cannot verify an implementation if you don’t know its intent. A specification is the textual or symbolic description by which different readers arrive at the same mental model. In distributed teams, software has long relied on PR review, ADRs, box and sequence diagrams — this is fundamentally the sharing of intent. You must know the developer’s intent before you can review their outputs. This is not new, it’s just now more urgent.

The genuine unlock of specs is that they literalize the behavior you need to verify. Once combined with simulation environments — headless browsers, terminal puppeteering, API smoke tests — your specs become the instructions for agentic verification *after* agentic implementation. The loop closes not at the generation layer, but at the verification layer.

Beware Reflexivity

These two variables don’t stay independent — they affect each other over time. Shipping too quickly raises C_v as verification debt accumulates. Higher C_v in turn raises future C_i — the agent’s implementation speedup erodes as the codebase becomes harder to reason about, harder to test against, and bad patterns get committed and cargo culted. This is the mechanism by which agentic coding gains can crash down to earth, trash a codebase, and turn a team against AI coding tools entirely.

But reflexivity runs both ways. The virtuous path runs in the opposite direction: to properly ship faster, teams must aggressively use the low cost to implement new tooling as a lever to decrease the cost of verification.

Solutions: Making Verification Tractable

Three concrete approaches, in increasing specificity:

1. Simulation environments at full breadth

• Standard integration/automation/E2E testing practices are table stakes, but the agent’s reach is now wider.

• Headless browsers, Chrome DevTools protocol, pseudo terminals, API smoke tests, microVMs, containers — the agent can drive all of these.

• The question is no longer “can we automate this” but “have we specified what to automate.”

2. Making the runtime legible.

• E2E tests don’t capture everything: service startup timing, internal program state, functional SLAs, “no UI interaction blocks for more than 2s”.

• An ephemeral, per-worktree observability stack — logs, metrics, traces — makes runtime behavior tractable to the agent.

• This is the difference between the agent knowing the tests pass and the agent understanding *how the system is behaving*.

3. Bespoke verification tooling is now nearly free.

• Personal example: while building a PTY proxy around several TUI tools, codifying system invariants and nightly fuzzing jobs cost only ~two extra hours.

• I implemented fuzzing with reproducible seeds and system state captures at failure.

• I can put my tool through more comprehensive testing than I could have ever justified without the agentic contributions.

• The economics of verification tooling have shifted — the agent makes it cheap to build guardrails and course correction, so the only remaining question is where those guardrails should go.

The New Discipline

Coding agents haven’t changed what good software engineering is, but they have changed where the leverage point is. The developers extracting the most durable value from these tools are the ones who have reinvested implementation gains into verification infrastructure. The question to ask before every agentic task is not “can the agent build this?” but “how will I verify what was built?”

If you want two immediate, concrete steps to improve your verification tools:

• After you run through a cycle of research-plan-implement, your agent must go through the implementation using TDD. Here’s a general purpose agent skill to do so.

• If you want an agent to go through a manual end-to-end test on your project, consider giving it the tools to interact directly. Here’s a skill for the agent to puppet a browser with Playwright. Here’s a skill for the agent to puppet a TUI with tmux.

I am hoping that this will be just the first of three core posts about the new practices in software engineering. To signpost properly where I think this is going, here are the three ideas that are changing how I think about software development:

1. Verification debt. (this post)

2. Agent legibility.

3. Compounding correctness.

Lots of interesting things in this post that I agree with. See also my very old post on the point of code review, and Cliff’s presentation at the May Agentics NYC meetup.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about how Nori can bring your company into the glorious AI future at norisessions.com.

Back at the end of March, we wrote about how we use coding agents to automate slide decks.

I like giving presentations. I hate making slide decks. I spend way too much time making slide decks. Most of that time isn’t actually that useful in terms of conveying information. It’s bs tasks like fiddling with positioning, aligning text, creating figures. Or it’s researching and pulling quotes or summarizing things that I already know but don’t have exactly ready for a deck. These things are necessary to have a good presentation -- I’ve seen many a research PhD slide deck to know otherwise -- but man all these little details bloat out to like 10 hours per deck.

We eventually figured out a hack to use coding agents to bring our average slide deck creation time to ~25 minutes of active work. The finished output is generally more aesthetic, more informative, and more accurate than anything we could do ourselves by hand. Certainly not in the same timeframe.

We wanted to see just how far we could push this framing. Could Claude Code make an entire video?

I was pretty impressed with how this whole experiment turned out! We wrote up a skill for all this here, in case you want to try doing the same.

What the fuck?

My first thought was that I needed to re-login. I run a ton of agents in parallel most of the time, so my instinct was that this was just a really really weird limit error. I vaguely knew that Anthropic was thinking about pulling Fable off the subscription plans so I switched to the API. Still nothing.

My team built a custom rust agent client, it’s pretty great. But my next thought was ‘o shit the harness bricked’, and I started poking around in Rust, which is a language I barely know even though I’ve ostensibly written tens of thousands of lines of code of it. At which point my friend went ‘the government banned fable.’

What the fuck?

But it’s true.

The US Gov directed Anthropic to disable access to Fable and Mythos to any foreign national anywhere in the world, including those in the US, including Anthropic employees. This is an impossible ask, and the Government knows it, so Anthropic has disabled all access to Fable/Mythos.

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Anthropic models will not be affected.

…

We received the directive from the government today at 5:21pm (ET). The letter did not provide specific details of its national security concern. Our understanding is that the government believes it has become aware of a method of bypassing, or “jailbreaking” Fable 5. We reviewed a demonstration of this specific technique being used to identify a small number of previously known, minor vulnerabilities. These vulnerabilities all appear relatively simple, and we have found that other publicly-available models are able to discover them as well without requiring a bypass.

…

We have reviewed a report that we believe is the basis of the government’s directive and validated that the level of capability displayed there is widely available from other models (including OpenAI’s GPT-5.5), and is used every day by the defenders who keep systems safe. We will share more details over the next 24 hours.

…

We are complying with the government’s legal directive and are removing access to Fable 5 and Mythos 5 for all users. However, we disagree that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people.

A few thoughts about this.

• Up front, I’m extremely conflicted.

• I am an AI doomer most days. Having trained many many deep neural networks in my time, I have a deep appreciation for the ways in which optimizers can go wrong. We optimize what we can measure, not what we actually want to achieve. We hope and pray that these are the same thing, but they often aren’t. We want to build good products, but we don’t know how to do that, so we optimize for engagement. We want to teach kids how to read and write, but we don’t know how to do that, so we optimize for test scores. We want to improve the economy, but we don’t know how to do that, so we kill thousands of whales and let their corpses just rot on the docks. AGI / ASI systems are optimizers, and optimizers can really be extremely dangerous in ways that are extremely difficult to predict, because in their efforts to optimize for what we can measure they optimize away from the good. Corporations are ALSO optimizers, so of course they are optimizing for ‘get money as fast as possible’ — the thing we can measure — despite many of the people building it being like ‘hey yea this is really dangerous,’ which is of course a fantastic parable for the whole AI alignment debate.

• But also, there is a massive shadow hanging over this whole thing. If any other government in the history of the United States took this step, there would be good reason to at least give that government the benefit of the doubt. But this government has shown itself to be petty and corrupt in ways that continue to completely and totally astound me in its openness and creativity. Is this coming from an actual desire to regulate AI? A better question: does anyone in this government who knows anything about AI actually have the ear of people who make these decisions? I would bet against it!

• Anthropic and this admin are very famously not friends. I am biased, but from my perspective, Anthropic tried their hardest to integrate with the DoD and work with the military, and as a result Anthropic models were used inside highly classified systems. And the Trump admin responded with a highly publicized attempt at corporate murder by declaring that Anthropic was a supply chain risk and that no one who works with the government is allowed to use any Anthropic models (which is basically everyone). All this despite continuing to use Anthropic models for military operations for the next 6 months, which included the entire war in Venezuela and the war in Iran. The Chinese LLMs aren’t even declared supply chain risks! Anyway this became a very public thing, and the far right twitter arm decided that because the Trump admin was trying to destroy Anthropic, Anthropic must be woke, and anything that is woke must be destroyed, so the Trump admin is right to destroy Anthropic.1 So is this admin trying to properly regulate harmful AI? Or do they see this as an opportunity to give a perceived cultural enemy a black eye?

• Meanwhile, Anthropic’s competitors have friends up and down the administration — the Kushners are heavily invested in OpenAI, as an example.2 So another way to read this is that this is an opportunity for other labs to give Anthropic a black eye. Fable is, by all accounts, an incredibly strong model. Very convenient that it’s no longer available for consumers, especially right as Anthropic is about to IPO.

• The problem with spoils system politics is that it makes the optics of everything suspect. We spent decades as a society getting to the point where we decided not to do that, and now that trust is gone. Again, a very long shadow.

• As a brief aside, I am once again extremely disappointed in the myriad of Silicon Valley people who angrily argued that a Democratic led government would ‘pick winners and losers in the AI race,’ who are now completely silent or defending the actions of this admin. I cannot help but feel that that previous posturing was just a machiavellian play for power, which has just been the worst feeling in the world. The tech industry does not have a great reputation with the rest of the world right now, not in the least because AI leaders keep talking about how AI is going to destroy everything and cause mass layoffs. It would be great to have someone with enough of a backbone3 to stand up for the principle and say ‘hey, this is kinda fucked!’

• This was announced on 5:21 PM on a Friday. Sorta a suspicious time. Whenever someone does something intentionally on a Friday evening, my first thought is ‘o, the markets.’ See, if you want to do something that is likely to be really bad, you announce it on Friday evening so that there is some amount of time for the stock market to settle during the weekend in the hopes that it won’t immediately tank everything. This isn’t the first time the Trump admin has tried this trick. From Claude (yes I’m aware of the irony):

  The most-cited compilation comes from the research firm The Kobeissi Letter, which catalogued a series of major geopolitical and trade announcements that landed after futures markets closed Friday or early Saturday, giving the weekend to absorb the shock. Their list reportedly includes strikes on Iranian nuclear sites on June 21, US military action against Caribbean drug boats on September 1, a 100% tariff threat against China after market close on October 10, the closure of Venezuelan airspace on November 29, military action in Nigeria on December 25, and direct strikes on Iran on February 28, 2026. They also flagged the corporate angle: on August 11, 2025, the administration announced an Intel deal after weeks of public pressure on CEO Lip-Bu Tan, again structured to land outside active trading hours.

• Why might this be a volatile decision? Well, a huge part of the AI boom is the idea that there will be ongoing demand for computer intelligence. The debt, the build out, the datacenters, the stock market runs on every part of the AI chain from GPUs to memory to disk to server racks. Alllllll of that is predicated on the idea that all of this is going to be worth trillions and trillions of dollars. And by all accounts, it seems like it is. Or at least, was on track to be. You know what will put a spanner in the build out of a multi-trillion dollar data center investment? The realization that at any point, the government will unilaterally cut off access to everyone, and the datacenters will be worth squat. Some people over on HN and Reddit are already talking about how this represents the high water mark for what the government will ‘allow’ people to access. You can have all the demand in the world, and it won’t matter a lick if the government just won’t let you have it.

• Speaking of the HN/Reddit folks, lots of people are gleefully cackling about how Anthropic got what they deserved for their ‘marketing stunt’ with Mythos. As I’ve said before, this isn’t the first time we’ve had an AI CEO argue that something is ‘unsafe’ for personal gain.

  Anthropic haters have roundly condemned this move as mere advertising and theatrics. OpenAI did the same “too dangerous to release” song and dance for the awesome, world ending AI that was GPT-2.

  Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code. We are not releasing the dataset, training code, or GPT-2 model weights. Nearly a year ago we wrote in the OpenAI Charter: “we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research,” and we see this current work as potentially representing the early beginnings of such concerns, which we expect may grow over time.

  I’m poking fun a bit here, but it’s worth noting that OpenAI’s concerns were actually pretty spot on. In the years since GPT-2 released, we’ve been absolutely flooded by AI slop that has pulled our collective ability to understand reality apart at the seams.

  I kinda believe that Fable is the real deal, and I kinda trust Anthropic when they say that they are concerned about the security risks of a widespread Fable release. But the top voted comment on the relevant HN comment thread is “Finally they will pay for all the scaremongering they been doing to sell their models as something so much ahead of all else.” Now no one can use Fable so…got em, I guess? Of course, this is quite possibly the greatest advertising for Anthropic you could possibly imagine. If you take the government at face value, the models are so good they literally can’t be used!

• Re: taking folks at face value, some people are also like “When you spend a lot of time telling people how dangerous your products are, people who have the power to keep dangerous products off the market might listen.” And to be honest, it’s a good point! Bernie Sanders, the most AI pilled member of Congress, keeps making policy videos that are just him reading off quotes from AI CEOs and being like “see?!” Which, of course, this brings us back around to the first thing: maybe these things really are unsafe and should be regulated. Of course, no AI CEO worth their salt is ever going to say something like that again, if they know the risk is being shut down.

• I want to end with another HN comment, that I thought was particularly close to how I feel.

  So many comments here missing the big picture, and just gleefully pointing out that Anthropic got what they deserved, or that this is the natural culmination of some kind of marketing stunt.

  The real story here is that this may be the beginning of governments restricting the availability of strong LLMs to the public, to you. Fable was the strongest model on the market, and the US government has told you you can’t use it (technically, only if you’re not a US citizen, but in practice, even if you are). If you think the solution here is going to be open source Chinese models and / or running on your own hardware, think again. Do you think China is going to allow the strongest LLMs from companies within its borders to be open source a year from now when they have Mythos capabilities, if the US government is keeping the strongest American models back? Unlikely. These are heading in the direction of being powerful cybersecurity weapons and it will be in the interest of nation states to restrict and control them. In 2 years time, I would be surprised if the strongest LLMs are available for general use at all.

  Will we be the poorer for that, or will we be safer? I think poorer, because I hate being told what technology I can and can’t use, but I’m not certain. Maybe you think the government should restrict strong LLMs. Maybe you don’t. But either way, this is big news and a rubicon has been crossed and a precedent set. That’s true even if the motivation for this is just the government settling scores with Anthropic.

Well said.

There are maybe a dozen “computer use” tools out there that promise to let a coding agent run a computer. These tools give the agent access to mouse actions and screen capture. The agent can only see the screen when it actively uses a screenshot tool. The agent can only move the mouse by inputting precise xy coordinates.

Imagine doing this with a person. I’d sit in my room, take a picture of my computer screen on my phone, then run downstairs. I’d show some the picture to some other guy who’s for some reason locked in this experiment with me, and I’d say ‘QUICK give me the x/y coordinates of this button’ as I frantically point to the like button on instagram on my phone or whatever. And this guy would say something like “idk, my vision isn’t very good because I mostly only know how to read and not much else1 but maybe like 1432x2384?” And then I run back upstairs to my computer and type in those numbers and see the mouse move to what I hope is the right spot, and if its not I’ll do the whole thing over again.

All this to say, computer use agents are mostly incredibly slow and aren’t very good and eat tons of tokens. Neat demos. Not useful for real work.

The gold standard is obviously an AI agent that can use a computer the exact way a human can, so you can automate any task with just a sentence. The next best thing is an AI agent that can sit in the background of a human run browser session, that can take over at any moment in order to manipulate the web page and directly examining network requests. The former isn’t really possible yet. But you can set up the latter with a tool called Playwright.

Playwright is a browser automation tool. It allows you to script interactions with a chrome derived browser. It (and selenium before it) were originally designed for integration testing on the web. They have found new life in the AI era as the primary way we allow agents to use and explore web pages. Because tools like playwright are scriptable, the agent can interact with them through programming languages like Python. No need to take screenshots and manipulate the mouse with coordinates. The agent can interact directly with the HTML.

Playwright and other tools are normally run in the background, without a visible UI. But you can also just run it like Chrome and interact with it like any other web browser. If you spin up playwright through a coding agent session, eg Claude Code, you have a fully functional browser that can be run by an AI agent at any time. Just switch to the terminal tab running Claude Code or whatever and have the agent take over.

Below, a few examples of how I’ve been using playwright for all sorts of browser automation.

Fixing a broken luggage damage claim form from a budget airline on the fly

I recently flew from Paris to NYC on a budget airline. I checked a bag, the airline damaged it, tale as old as time.

Of course this airline doesn’t have any person that you can talk to about this. Instead, they have an online form where you can submit your claim. Credit to the airline, the form mostly works. You can tap out a complaint into a text box and it will submit the complaint. But if you try and attach any photos, say, of the luggage damage, the system just borks. Specifically, the ‘submit’ button remains greyed out and there’s no way to actually upload the supporting files. Probably not intentional.

Since I was running my browser through a Claude Code managed playwright session, I just tabbed over to the terminal and asked Claude to take a look at the page, figure out why the button wasn’t working, and then submit the files. Since Claude had access to the full html, js, console, and network requests, it quickly figured out that:

1. after each file upload a modal should appear that lets the user set the type of upload;

2. That modal wasn’t appearing, resulting in the form remaining disabled;

3. It was possible to simply set the form button as ‘enabled’ and submit the data;

Which it then did. It was even able to validate that the upload went through successfully because it could read the network request statuses.

Automating social media cleanup

A few years ago, after Facebook Messenger added a message unsend feature for what I assume are gdpr compliance reasons, I built a little browser extension called ‘Shoot the Messenger’. It was reasonably popular, with tens of thousands of downloads. The extension would automate your browser, hooking into messenger css selectors to automatically scroll through a messenger thread and sequentially unsend every message in the thread.

This was a nightmare to maintain.

Every time messenger changed their UI, all the css selectors would break. So maybe once a month I’d get a dozen emails and bug reports that the extension wasn’t working right.

With an agent powered playwright, the same functionality is trivial. I can open any messenger thread, tab to the terminal and tell the agent to record my actions, then delete a few messages and tell the agent to do the rest. Since it’s programmatic, I can basically just leave the script on without spending more tokens and go do other things while messenger cleans itself up.

LinkedIn scrapes

LinkedIn is notoriously hard to scrape. They aggressively tamp down on most bot-like behavior. Unfortunately, LinkedIn is also rather difficult to be on as a human. It’s, uh, a lot of AI generated content, to say the least. “Here are five things I learned about entrepreneurship when I stubbed my toe last week.”

Anyway, by this point you probably get the pattern. Spin up LinkedIn through playwright, show the bot what I want it to do, and then go get a coffee while it figures the rest out.

I found all of this compelling enough that I spent last weekend putting it together as a little side open source side project, which you can access here.

At it’s core, it is just a playwright session on the right and a little Claude Code session on the left. The Claude Code session is given system instructions to understand how to access the playwright session. Everything else is just using the browser like normal…except for when you want to automate something. Just as a silly example, asking the bot to try and find the emails from the folks who had top comments on HN:

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about Nori at noriagentic.com.

I run Claude Code1 in YOLO mode.

I know I shouldn’t. I know it’s a terrible idea. But I just cannot stand sitting there and hitting enter every 30 seconds because Claude wants to read a file.

After mindlessly hitting enter for the ten millionth time on some prompt, I snapped.

I aliased Claude to claude —dangerously-skip-permissions

And even though I’m now free from the burden of having to babysit the agent, I have a new fear.

One day, I don’t know when, Claude will rm -rf my entire computer.

In the past, I’ve said that coding agents are a lot like a really smart, totally untrained intern. I still basically believe that. Even though models have gotten better over time, they still come to every session completely fresh without any context on you, your company, or your engineering practices. Would you give a new hire intern unlimited access to your local machine? You know, where you have all your ssh keys and your contracts and a ton of personal data and maybe photos of your license/credit card/passport?

The fundamental tension is that we want agents that are highly autonomous, easy to set up, and really secure; and in practice, we can only pick two.

• You can install a frontier coding agent (Claude Code, Codex) locally and set up all the MCPs and it will be super easy because local dev is the happy path, and then the agent will one day nuke your email inbox.

• You can install a highly controlled permission-gated agent locally, get all the easy set up, and then spend your whole day hitting enter on permissions prompts one at a time.

• Or you can take a month to build out an isolated sandbox service to run Claude Code as a background agent. The agent works, but you’re too deep in docker networking configs to really enjoy it.

The vast majority of people are in the [powerful, easy, insecure] part of the embedding space. Mostly by choice — the average consumer of Claude Code probably couldn’t figure out how to set up a secure sandbox and definitely doesn’t want to use a limited agent.

There are some things that you can do to make the local experience slightly more secure while minimizing the amount of “Yes, and don’t ask me again” you need to do.

• Turn on /sandbox. It's OS-level isolation (bubblewrap on Linux, Seatbelt on macOS) that confines writes to your working directory and routes network through an allowlist proxy.

• Add explicit deny rules for the stuff you never want touched — ~/.ssh, ~/.aws/credentials, .env, id_rsa, .npmrc, ~/.netrc — in your settings

• Write good skills and agent configs, and don’t let your conversations go through too many turns.

Anthropic even shipped an entire permissions mode that is meant to address this complaint.

Claude Code users approve 93% of permission prompts. We built classifiers to automate some decisions, increasing safety while reducing approval fatigue.

But these will always be bandaids. The fundamental tension of a coding agent is that you want to give it access to things. It needs to be able to write and read files in order to do the most basic tasks. Unfortunately, that alone is enough to break out of most guardrails. Your filesystem is a core part of the agent config, so you should assume that anything on your disk is fair game.

I think the only actual way to have your cake and eat it too is to go the sandbox route. Ideally sandboxes in the cloud, far away from your local files, on boxes that you can tear down without hesitation because they don’t have anything precious in them. Yes, setting up background agents is a pain, but that will get easier over time as people figure out how to implement the infra. And you get more than security — agents that live on cloud sandboxes can be accessed remotely from a ton of different interfaces, and can be triggered by automatic events like bug reports or customer inbound.

For our agent environments (Nori Sessions), we went with a firecrackers micro VM provider for the base machine.2 These are extremely easy to start and kill, which makes them perfect for building agent runtimes that are constantly refreshing themselves. The VMs themselves don’t really have state. Instead, each VM has a setup script that clones all the relevant repos and sets up all the dependencies, so each agent session runs on a clean box. The agent can basically destroy the machine and it won’t matter, we’ll just recycle the thing.

The actual keys are never kept on the same box as where the agent is running. Instead, we proxy between the agent runtime and the outside world. Whenever we see certain credential placeholders in anything going outbound, we can trigger conditionals that determine whether we should inject the real credentials. The end result is that the agent thinks it has access to everything, but in practice it’s tightly controlled.

This isn’t the first time I’ve harped on the benefits of cloud agents. I also wrote about them in my post on AI enablement.

I think there’s a big gap in the market for managed agent runtimes (also known as ‘background agents’ because they can run in the background on the cloud). Everything else — org level skills, for eg — is just a band-aid. I’ve said in the past that for coding agents, the filesystem is the config. That means that you have to control the entire filesystem to get a consistent (and secure!) experience. It is simply not enough to assume that everyone in the org will learn not just how to use AI, but also how to use a CLI and how to use Bash and how to use git and how to use the million other tools that are required to get a coding agent off the ground. And even that may not be enough.

…

I think most teams should go for a fully managed service. Devin has been in the space for a long time. I think Twill is a relative newcomer that also does something like this? But there aren’t a ton of other services that actually follow the Ramp-Inspect / Stripe-Minions model. So when we ran into this problem ourselves, we weren’t happy with most of the other options out there and set out to build our own, which we now sell as an off the shelf option with a lot of customization flexibility.

I have to say, having a tool like this is pretty incredible, and I’m really glad we built it. More than 30% of our PRs are shipped entirely through slack, and we have all sorts of really cool automations for things like bug triage to newsletter writing. But all of that takes a lot of maintenance, and we spend literally all day just thinking about how to make this one thing awesome, and it would be impossible to do that if we were building in, like, healthcare or manufacturing instead.

It’s hard to explain how transformative these are for a company, but once you have them you cannot really go back. My best metaphor is that a good background agent setup is much more like hiring an extremely capable engineer than it is like interacting with a tool in a terminal. It is very natural and intuitive to interact with these things like a person, which short circuits a lot of hard learning curves and enables a lot of collaboration.

And, of course, they are secure. Way way more secure than running a local agent, or giving a new hire intern access to your personal laptop.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about Nori at noriagentic.com.

Most people probably don’t know that I also have a long-standing interest in religious studies. I’m not particularly religious myself, mind you — I tend to swing between atheist and agnostic, in case it matters — but I have a lot of respect for religious contributions to philosophy, ethics, and culture. Reading religious texts is a great way to understand how entire civilizations organize themselves. And I also increasingly believe that some kind of faith is necessary to live a self-fulfilling life. Ethics is hard, personal responsibility for ethical choices is an immense burden. What a gift, to know what is moral or not based on the contents of a book or the teachings of an elder! It’s a shame that so many of my college-educated FAANG-employed friends never bothered opening the Quran or the Bible.

Anyway, this has been a particularly great week for me, because my two favorite hobbies randomly and unexpectedly intersected when the Pope released his encyclical on AI.1

Encyclicals have a long history, but you can think of them as letters to the public that outline the Catholic Church’s position on a subject. They’re not exactly the same as the word of God, but they are important entries into the long and storied Catholic philosophical tradition that includes Aquinas and Augustine, as well as Dante and Copernicus.

The encyclical on AI (officially called Magnifica Humanitas, or Magnificent Humanity) is a long document.2 It’s also, as you might expect, a nuanced document. If you were expecting a clear “AI is a force for good / evil”, you will be sorely disappointed. Leo has a fantastic grasp of the potential and risks of AI as a technology, and avoids the polarization that has plagued the discourse on basically everything.

One of the longest threads in the encyclical is on the question of what kind of technology AI will be, expressed through extended metaphor.

Will AI be like the Tower of Babel? Will it be an exclusionary project founded on the desire to surpass humanity? Will it be doomed to fail, to sweep away those who built it in a catastrophic fall rivaling the flood? Or will AI be like the walls of Jerusalem? Will it be a protective and defensive force that can shelter those who need it while encouraging the fulfillment of all mankind?

Despite being an encyclical about AI, the Pope is not particularly interested in AI as a technology. Technology does not have animating spirit. The ethical parameters of a piece of technology are defined by those who build it. What are their goals? What is their vision of the future?3 IMO, Magnifica Humanitas is less about AI and more about Silicon Valley.

The problem with AI is not AI. It is the transhumanism. Or the belief that money is a terminal indicator of worth. Or the belief that humans should be replaced by AI. It’s the culture, stupid.

92. In his Encyclical Laudato Si’, Pope Francis denounced the growing dominance of a technocratic paradigm [119] in our globalized world: the tendency to let the logic of efficiency, control and profit alone shape personal, social and economic decisions. This makes it clear that technology is not simply a tool. When it becomes the standard by which everything is judged, it begins to dictate what matters and what can be discarded, reducing creation to an object of exploitation and human beings to mere cogs in a system driven toward ever greater efficiency.

…

94. The danger of humanity becoming a victim of its own achievements was already clearly recognized by Saint Paul VI, who warned that “the most extraordinary scientific progress, the most astounding technical feats and the most amazing economic growth, unless accompanied by authentic moral and social progress, will in the long run go against man.” [121] For this reason, technological progress — valuable in itself — requires careful discernment of the anthropological vision that guides it and the ends it pursues. If technological development advances without a corresponding ethical and social progress, the result may be an increase in means without a growth in humanity: “having more” without “being more.” In such a scenario, there is a risk that individuals will be evaluated principally according to the outcomes they produce. [122]

…

115. In an attempt to shed light on the cultural assumptions accompanying the ongoing digital revolution, I would now like to turn our attention to certain currents of thought that interpret progress as surpassing the human condition, and which are often grouped under the labels of transhumanism and posthumanism. These perspectives form the ideological background present in some centers of technological power and occupy the collective imagination in a simplified form, especially in the media and on social networks. They tend to foster enthusiasm for new technologies through a futuristic vision of an “enhanced human being” or “human-machine hybrid.”

…

117. From the perspective of the Church’s Social Doctrine, the key issue is not the use of technology as such, but the vision that underlies it. If the human being is treated as something to be perfected or surpassed, it becomes easier to accept that some lives are less useful, less desirable or less worthy. In the name of progress, “necessary sacrifices” may begin to be justified, placing the burden on the most vulnerable in pursuit of a supposed optimization of the species. In this regard, the aforementioned warning of Saint Paul VI retains great foresight: indeed, scientific and technological advances, when detached from moral and social progress, end up turning against humanity. For this reason, a clear distinction must be made. It is one thing to integrate technology within a human-centered, relational vision; it is quite another to be guided by an outlook that devalues human limits and promises a purely technical form of “salvation.”

So is AI going to be the Walls of Jerusalem, or the Tower of Babel? Let’s ask any of the thousands of people in the Bay who believe they are literally “building God.”

One of my favorite quotes from the encyclical is about the ‘absolutizing of intelligence’:

In reality, elevating any single dimension of human existence to an absolute is always a mistake. Indeed, disorder does not arise only from scarcity; even unchecked growth can give rise to impoverishment. In an ecosystem, balance is disrupted when one species expands at the expense of others; in human life, something similar occurs when one faculty claims to be the measure of everything. Thus, intelligence, when absolutized, overshadows other essential dimensions of life, such as affection, the will, commitment and relationships.

Which really stands in stark contrast to various AI leaders who keep calling humans “meat computers”.

Musk:

We are all dumb meat computers compared to digital superintelligence.

Karpathy:

One day, frontier AI research used to be done by meat computers in between eating, sleeping, having other fun, and synchronizing once in a while using sound wave interconnect in the ritual of "group meeting". That era is long gone.

Ellison:

The brain is very specialized, so are the AI models. But we’re not building a 20 watt meet computer. We’re building a 1,200,000,000 watt AI brain.

I think Leo is seeing and reacting to the same cultural trends I observed here (and to some extent here and here), but at a larger scale.

I want to reflect briefly on the surge of nihilistic discourse that seems to increasingly plague the tech community. This particular disease goes by many names — ‘black pill’, ‘stoicism’, ‘pragmatism’. But don’t let that confuse you. These are all forms of the same thing.

The core of this mindset is a logical fallacy:

• The speaker has realized that ‘everything is made up’, that ‘there are no rules’, that ‘you can just do things’;

• therefore having values, beliefs, or morals at all is misguided, naive, cringe, and uncool.

A classic example that exemplifies the mindset is something like “but it gets clicks/views/engagement.” Another one: “this is just how the world works.” A third: “it’s not my problem.”

The nihilism and pursuit of wealth at all costs doesn’t exist in a vacuum. It comes from the dark side of the entrepreneurial spirit, the same force that encourages creating gambling addiction or using rage bait marketing or blatantly violating laws to get a competitive edge. AI is a multiplier; in the hands of unscrupulous people, AI will be unscrupulous.

In some sense, Magnifica Humanitas is concerned about the sorts of things alignment researchers have been beating the drum about for ages. Chris Olah, an Anthropic founder and a leader in AI alignment, was invited to speak alongside the Pope for good reason. But the difference is that the researchers think they can solve the problem if they embed ethics into the technology. The Pope suggests that we embed ethics into the society that creates the technology instead.

Can AI Create New Knowledge

Two open questions, both extremely important.

First: is knowledge created or discovered? In some domains it seems obviously discovered — bacteria and viruses wreaked havoc well before germ theory could explain why, the arrangement of the stars and planets are agnostic to our telescopes, and the higgs boson continues to do…whatever it does regardless of our ability to observe it.

In others, it’s less clear. Mathematics is the study of axioms, a seemingly endless game of pushing a particular set of contrived rules as far as possible. Is the Pythagorean Theorem created or discovered? Any given proof of the Pythagorean Theorem is arguably invented. But also, in some sense, the relationship between sides of a triangle exists in the ether, waiting to be plucked out of the chaos by the first Greek guy with a ruler and compass. And proofs themselves are often just applications of existing knowledge in other places. Is that invention? Or do the laws of mathematics already exist, and mathematicians are merely filling in parts of the map?

Second: can LLMs create new knowledge? This seems to be the last remaining stronghold of the AI denialists. “Yes, fine, AI can create art and create music and write essays (though Sam Kriss may murder me for saying so) and build software out of nothing at all, but yawn wake me up when the AI can discover fundamental truths about the world.” Nevermind that most people cannot meet that standard, but whatever.

Notice that the second question is intimately tied to the first question. If creating new knowledge is an act of consciousness, a leap of ingenuity that cannot exist from any previous understanding of the world, then LLMs have not and arguably cannot create knowledge. But if creating new knowledge is an act of discovery, of interpolating between known points on a map and finding the points that are unexplored — that is, if creating new knowledge is merely a particularly exhaustive exercise in search — then of course LLMs can create new knowledge, it seems trivial to argue otherwise. LLMs are fantastic at search, and as I’ve written in the past, all problems are search problems.

In some ways, both of these questions are debates about semantics. What do you mean by knowledge? There’s an entire arm of philosophy (epistemology) dedicated to the question of what it means to know things! People have dedicated their lives to trying to understand this meta question! When Twitter user CiceroClone3524 smugly tweets “LLMS CAN’T CREATE KNOWLEDGE”4 without first providing a treatise on what knowledge even is, you should just treat it like the incoherent shitpost it is and move on.

This month, an LLM solved an open problem in discrete geometry.

For nearly 80 years, mathematicians have studied a deceptively simple question: if you place n points in the plane, how many pairs of points can be exactly distance 1 apart?

…

Today, we share a breakthrough on the unit distance problem. Since Erdős’s original work, the prevailing belief has been that the “square grid” constructions depicted further below were essentially optimal for maximizing the number of unit-distance pairs. An internal OpenAI model has disproved this longstanding conjecture, providing an infinite family of examples that yield a polynomial improvement.

…

Surprisingly, the key ingredients of the construction come from a very different part of mathematics known as algebraic number theory, which studies concepts like factorization in extensions of the integers known as algebraic number fields.

This was immediately met by cries of disdain. For example, Gary Marcus:

Clearly impressive. But as with so much else, it should be viewed with skepticism.

…

[Quoting Cal Newport] He noted that though the conjecture is old, those who have worked on it have largely shared Erdos’s original belief that it was true and therefore focused on trying to solve it. What the LLM-based tool did instead was to systematically apply and extend existing techniques in search of evidence that the conjecture was false.

…

[Quoting Tim Gowers] It has an encyclopedic knowledge of mathematics, and it does not have to worry nearly as much as we do about time management, so it is good at finding surprising connections.

And so on.

Without commenting on whether or not this is “new” knowledge,5 it’s worth pointing out just how much ground the AI denialists are ceding. Science is about building on the shoulders of giants. Virtually everything we know is derived from combining the ideas of our forefathers. Hell, almost every AI paper is about applying some idea from some other field to AI, and all the other AI papers are about applying some idea from AI to some other field. If a novel mathematical proof isn’t new knowledge, the term “knowledge” has ceased to refer to anything that corresponds to reality. At that point, just pack it up — humanity never had any hope anyway.

The Big Labs get into Consulting

I think “forward deployed engineer” is my favorite euphemism of the last decade or so. As far as I can tell, the only reason this term exists is because a significant portion of Silicon Valley will refuse to accept that they are consultants, but if you wrap their title with the term “engineer” suddenly everything is ok.

What does a forward deployed engineer do? Well, they are “deployed” to client teams in order to help those teams solve their business needs, often by coding. What does a consultant do? Well, if they work at say, Accenture, they are deployed to client teams in order to help those teams solve their business needs, often by coding. “No no, I’m not a consultant, I don’t work with slide decks and things. I’m beyond such mundane software as the Microsoft Office suite. I’m an engineer.” Uh huh. I mean I agree that management consulting probably uses slide decks, but my buddies at McKinsey write code too so IDK. Hell, with coding agents, everyone writes code these days, especially the consultants.

To be clear, I have nothing against consultants or consulting. It’s obviously necessary, especially when there’s a new technology that lots of people want to use but few people have expertise in. I just think it’s funny that the Valley had to invent a new word for it. Kinda in line with the whole “package an existing product in a new name and sell it as revolutionary,” though that is more often done to avoid regulatory scrutiny (looking at you, prediction markets / crypto / ride shares / whatever software landlords use to coordinate rental pricing increases).

Anyway, my point is that FDEs have been around for a while. If you are having trouble with the middle of your sales funnel — folks are maybe signing up for your product but not getting the full use of it, maybe not getting upsold effectively — sending out one of your employees to help them get set up is common sense.

This is why Microsoft hires Solution Engineers:

• Drive technical sales by using technical demos, proofs of concept, and technical architecture accelerators to influence solution design and enable deployments.

• Lead architecture sessions and technical workshops to accelerate Azure adoption.

• Build trusted relationships with platforms leads to co-design secure, scalable solutions.

And why Oracle hires field engineers, and why Salesforce, Atlassian, Intel, Google, all have customer success engineers, and so on. Apparently IBM had “customer engineers” way back in 1942. The actual term “FDE” got popularized by Palantir, and since then I’ve seen Fortune 500s and Series A startups alike advertise for FDE roles. But, you know. It’s really just another name for consulting.

This is why I was somewhat surprised about the news that OpenAI and Anthropic were both starting consulting arms. Not that they were starting those arms, mind you, that makes tons of sense, see above. Rather, I was surprised by the news — the breathless takes from an entirely too credulous social/media that this was further evidence of all economic activity being subsumed by the big labs.

Guys. Guys. Come on. Get off the Internet for two minutes, please.

OpenAI is starting a consulting arm because people keep going “I want to AI more” and then OpenAI goes “here is your AI” and then people go “thanks but I have no idea what this is, also can you make it hook into my RBAC provider?”

AI tooling was built for consumers first, so it is easy to forget just how involved enterprise software is. “Just npm install -g claude” you fool. You absolute fool. The average enterprise user cannot just npm install -g claude. Hell, they can’t even install npm! They have to request approval to get access to the site that hosts an internal version of npm, then request approval to download npm, then set npm to point to an internal registry, then get approval to download an internal version of Claude off that registry, and then finally have that version of claude point to some internal enterprise exclusive Claude deploy. Someone has to maintain all of those security boundaries, someone has to explain how the AI works within those boundaries, and all of the people who know how to do that currently work at the big labs because AI has only existed in its current form for like 8 months. If you made wild predictions about how “Microsoft sending consultants to help enterprises onboard onto Windows” meant that MS was going to take over the world, you would’ve been right for about 5 years from 1995 to 2000 and then very very wrong for the next two decades.

That’s not to say that OpenAI and Anthropic and so on are bad products, nor even that they won’t benefit from the expansion of AI usage. Microsoft undoubtedly made more money in absolute terms as computer adoption skyrocketed, and the same is certainly true of the big AI labs. But also, there were clearly many billions to be made on the back of the computer boom, and on the back of the Internet boom, and on the back of the mobile boom. The AI boom is very much in its infancy, and, contrary to popular opinion, the existence of these deployment arms — which are essentially high labor, white glove services companies — just underscores how early we are.

Other things:

• The Pope’s encyclical is a thoughtful and nuanced document, so of course most of the Internet spent most of the time debating whether or not the thing was written with AI. There’s good reason to believe parts of it were, or at least that some translations were done with AI. But also, this discussion completely misses the point. The Pope never says that using AI is bad! AI isn’t an all or nothing thing, and if used responsibly it is obviously extremely valuable. The problem with AI writing is AI slop; the Magnifica Humanitas is obviously not that.

• SpaceX is IPO-ing. Matt Levine has the best take, as usual:

  The deal, with SpaceX, is that Elon Musk runs it however he wants, and he does weird stuff, and you have to trust him, and if you don’t like it you can’t complain. When SpaceX acquired xAI a few months ago, did a special committee of independent directors approve the transaction? Did Musk recuse himself from negotiations? Was the price set by independent valuation experts using a rigorous process? Did outside shareholders sue to block the deal? Stop. Musk wanted SpaceX to buy xAI, so it did.

  SpaceX claims to have a TAM of the entire global market and then some. This is obviously ridiculous, but if you’re investing in SpaceX you’re obviously along for the ride. The bigger concern, in my mind, is SpaceX getting index funds to change their rules. TBD on how that all works.

• Mythos seems like it's beating the hype. From Anthropic: “Since then, we and our approximately 50 partners have used Claude Mythos Preview to find more than ten thousand high- or critical-severity vulnerabilities across the most systemically important software in the world. Progress on software security used to be limited by how quickly we could find new vulnerabilities. Now it’s limited by how quickly we can verify, disclose, and patch the large numbers of vulnerabilities found by AI.”

• Instagram was hacked because of AI, but not the way you might think.

  Step 01: Faking the Location & Initiating Support All the attacker needs to kick this off is your account username. Then, they hop on a VPN or proxy close to your city so Instagram’s security algorithms don’t suspect a thing. (You can quite easily get this from your public profile or “About” section or a hundred other ways.) Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.

  
  Step 02: That’s It Really, that’s it. The first proper zero auth password reset I’ve seen in production. There appears to be no additional check as to whether the email being given is actually something the user has used before. Once the AI sends the security code to the attacker’s email, the attacker passes it right back to complete the verification. The platform hands over a fresh password reset link, granting full ownership to the attacker.

  “The first proper zero auth password reset” is hilarious. Reminds me of the good old days when you could talk to “customer support” and have it promise you a car that the company would then be forced to pay out.

• Chipotle runs an AI model for customer service. Apparently it’s just a standard coding agent, so someone figured out how to hook into it for their coding work. “Free inference paid for by burritos.”

I’m worried that you read “everything I love that would be reasonable to put in a sandwich.” But that’s not what I wrote. I wrote “everything I love.” So, for 8 year old me, that included Costco taquitos, and fruit by the foot, and bologna, and ice cream, and lunchables (which kind? Does it matter?)

And you know what? That shit tasted awful. It made no sense. Threw most of it out, never made one of those sandwiches again.

Anyway, let’s talk about Dune Imperium.

My buddy Azraf got me this game as a birthday present 4 months after my actual birthday, because he knows I don’t like birthday gifts but I’m ok with random “here is a gift because you’re my friend” gifts that land on a totally non specific Tuesday. In the time since he got it, we’ve played the game maybe a dozen times with the friend group. The box says that a game takes about 60 minutes to run through. This is a lie. We have yet to finish a game in under three hours.

Dune Imperium is what you get when you take your favorite mechanisms from every other board game you’ve played, and stick it into a single game. It’s got the worker placement mechanics of Agricola or Viticulture. It’s got the deck building mechanics of Star Realms or Dominion. It’s got the betting / positioning mechanics of poker. It even has the random bullshit cards that you get in Catan.

It is, in other words, an “everything-I-love” sandwich as a boardgame. Except unlike my childhood culinary monstrosity, Dune Imperium actually works. It works really well. Last I checked, Dune Imperium was sitting at number 6 on the boardgamegeek rankings.

How? What kind of alchemy makes a game this convoluted actually fun?

My current best guess: the game has a sort of circular rhythm to it, which makes the whole thing function.

The rough structure of Dune goes like this:

• The game has ten rounds, and each round involves players going in a circle taking a turn until no one can do anything anymore;

• On your turn you play a card. Based on the card, some things happen, and you can place an agent somewhere on the board that doesn’t already have an agent on it;

• Placing an agent may require some resources (water, spice, coin), and will often give you some resources (water, spice, coin, military units, persuasion, card draw);

• Once you run out of agents, you can pick up some new cards from a shared market using any permission you have, and add them to your deck;

• Once everyone has finished, you evaluate who invested the most in military units — that person wins the combat for that round and gets some award;

• The round resets and the next player in the circle goes first…unless someone has 10 victory points, in which case the game ends.

It sounds like there are a lot of options, but unless you do something really stupid you’re pretty much on rails for most of your turns. You get some water. The next turn, the water lets you get some spice. The next turn, you turn the spice into units. The next turn, you deploy all the units and win a combat. Now you’re out of units and out of resources, so you start the cycle again.

There’s something very civ-coded about these mechanics. At each step it feels like you’re making some complicated decision; in practice, you are highly limited by what resources you have, what cards you have, and where your opponents place their agents. You’re basically forced to do something useful with your agents each turn. Good players will negotiate with each other and try to block other players, and there is some skill required in figuring out how many troops to deploy for a given combat (any troops you deploy will be removed for the next turn, regardless of whether you won or not, so figuring out what to bet is all about reading your opponents). But in general this is not a tactics-heavy game. In half of our games, players would simply pre-move before the other players had finished their turns, because they knew that new information wouldn’t change their play.

Instead, all of the actual skill of the game is in the long term strategy. Again, the game is only ten rounds. Almost all of the victory points that you need to win are doled out in the last three rounds. And basically every game I’ve been in has been shockingly close, which I think is due to a combination of the relatively short amount of time to build an engine and the amount of ‘guidance’ even new players have in terms of basic tactics. As a result, good players are constantly scheming about their last few turns from the very beginning of the game. Mostly, that ends up being about the deck building — that is, understanding how to value different cards based on your existing deck, your particular leader abilities, and your personal road to 10 points.

O, and the random bullshit cards. Did you think I’d forget to mention the random bullshit cards?

Despite being a pretty ‘crunchy’ rules heavy game, there’s a surprising amount of variance built into Dune. Some of that is built into the market-place drafting concept. Maybe you’re unlucky and all of the good cards keep getting bought up before your turn, whatever, that’s the draw. But when someone pulls a god damn bullshit card right as you’re about to win a combat that you spent four turns preparing for, that’s when you see red and start looking around for actual daggers.

Bullshit cards (in the game they are called ‘Intrigues’ but we have only ever called them bullshit cards) are special cards that you can pick up through the game that have a random effect. These things have extremely high variance. One card gives you two coin, which is nearly worthless after turn three. Another lets you get the military equivalent of 4 units, which you can choose to spring at the last second when everyone has committed their bets for combat. I’ve been in at least two games where someone top-decked a bullshit card to steal the win.

In a lesser game, that variance would be enough for me to drop off entirely. Crunchy games like this one depend on making their players feel smart. If you lose to random bullshit, you don’t feel smart, you feel cheated. But Dune somehow manages to to transmute that frustration into a desire to play again. I think it’s because in most games, everyone feels like they are one turn away from winning. That “so close” energy makes the game stick in your head, constantly turning over what things you could have done differently to change the outcome.

Of course, I would say that, because I’ve actually won the last few games I played. Some of my other friends have lost 5 games in a row. I think they are less enthused. The biggest issue with the game is that there is no real come back mechanic. When you’re behind, you’re just eating glass for the next two hours. It’s still possible for the lead to screw up, but in a competitive game you want to have options for forcing errors (again, see ‘make the players feel smart’). Dune doesn’t really have that, in large part because of the ‘on rails’ tactical feel to the whole thing. If a player buys a fantastic card off the market in the first turn or two, the game might just straight up be over. Which sucks, because of course you still have another 3 hours of game ahead of you.

Still, it’s a miracle a game with this many moving parts works at all. There are literally 5 different currencies to keep track of! I cannot begin to fathom the amount of play testing they had to do to get this game to feel even remotely balanced. It’s a testament to the designer that you can pursue a pretty wide variety of strategy and still come close to winning.

At the meta level, I find Dune fascinating as a piece of game design. I’ve long felt that there are only like 15 different kinds of game mechanics,1 and most boardgames are just mixing and remixing those into different combinations and themes — for eg, Seven Wonders is just the card passing mechanic from hearts, Sushi Go is just seven wonders with different scoring, etc. Most games need to settle on one mechanic in order to have any hope of being good, anything more ends up just being too much.

Dune joins Spirit Island as a rare exception that proves the rule. Also it’s the only game where you can play as a scrawny pingpong player that got dropped in the dessert.