Given how things are in 2026, I think it’s fair to say that that line of reasoning didn’t really go that well for them.1

The big issue is that it turns out there are lots of reasons why people will do things that are harmful to themselves. This is not exactly a brand new idea, at least in the common-sense understanding of things. Alcoholism is such a well known issue that supposedly an entire ethnic group developed a genetic aversion to the stuff, and history is littered with temperance movements and the like. But all previous attempts at regulating addiction were framed as ethical, pseudo-religious necessities. God created the world, but he also created alcohol, or something, anyway no you can’t have whiskey.

The regulations that sprang out of the mid-to-late 90s were grounded in science, which was a bit new, and was only really possible because science had caught up to our intuitions. We could point to causal mechanisms of addiction. Those mechanisms drove choices that these companies purposely made. Those choices made their products (cigarettes) more harmful to end users. And that harm was the basis for liability. The tobacco companies ended up having to pay massive fines ($200+ billion), while state and local legislation effectively banned smoking in most public places.

As a result of that work — both educational and legislative — we have a simplified but very well understood language for addiction. There are substances that can make someone behave as if they weren’t themselves, and these substances should be regulated.

It is massively helpful to have this baseline understanding of addiction in the general consciousness. Before, people would say silly things like “anyone can quit cigarettes, they’re just lying if they say they can’t” (while taking a drag of their fourth cigarette before breakfast). The language of addiction changed this. By making it clear that addictive substances changed a person’s brain chemistry, it became way more acceptable to shift the conversation from personal responsibility to corporate liability.

Of course, the language of addiction is very simple, and as a result there are a whole bunch of addictive things that don’t fit neatly into the language above. For example, gambling. I think basically everyone knows on some gut level that gambling can really screw someone up. And there’s lots of research to back this up. Just to pick one out of a hat, fMRI scans show that gambling really can change brain chemistry. But it’s not, like, a ‘substance’. There isn’t any obvious ‘thing’ that is changing brain chemistry. It’s the action of gambling itself that is a problem. And this really breaks some people’s brains, because how can an action, or a concept, or an idea change your brain chemistry as much as some good ol’ C2H5OH? As a result, gambling is way less regulated, and has crept into all sorts of weird places. Sports betting, prediction markets, loot boxes, gacha games. A lot of these target minors explicitly, because it’s not ‘addiction’.

In a past life, in my molecular bio days, I spent a fair bit of time doing addiction research.2 When it comes to addiction, there isn’t any special requirement for something to be a ‘substance’. You can get addicted to drugs. You can get addicted to fatty foods and to coffee. You can get addicted to visual stimuli (porn), or physical actions (a wide range of eating disorders), or even toxic relationships. Everything we do ends up being translated to protein pathways and electrical signals. Sometimes that warps our brains. And sometimes, people build things that warp our brains intentionally.

Last week, one of the most important cases about social media regulation wrapped up. From NPR:

A California jury on Wednesday found that Meta and Google were to blame for the depression and anxiety of a woman who compulsively used social media as a small child, awarding her $6 million in a rare verdict holding Silicon Valley accountable for its role in fueling a youth mental health crisis.

While the financial punishment is miniscule for companies each worth trillions of dollars, the decision is still consequential. It represents the first time a jury has found that social media apps should be treated as defective products for being engineered to exploit the developing brains of kids and teenagers.

Lawyers for KGM argued that Instagram and YouTube were deliberately designed to be addictive and the companies knew the platforms were harming young people, while the tech companies countered that their services cannot be blamed for complex mental health issues.

KGM’s legal team showed the jury internal documents from Meta in which CEO Mark Zuckerberg and other executives described the company’s efforts to attract and keep kids and teens on its platforms. One document said: “If we wanna win big with teens, we must bring them in as tweens.” Another internal memo showed that 11-year-olds were four times as likely to keep coming back to Instagram, compared with competing apps, despite the platform requiring users to be at least 13 years old.

Under questioning about those documents, Zuckerberg told the jury that keeping young users safe has always been a company priority. “If people feel like they’re not having a good experience, why would they keep using the product?” Zuckerberg said.

I think there are a lot of things you could say about who is to blame for the plaintiffs social media consumption habits. You could argue personal responsibility. You could say that the parents should have been more involved — the plaintiff started on the apps when she was 6 (!!!). You could even say that schools and teachers and peers all played a role. But all of these assume that social media itself is, inherently, a risky thing, something that needs to be managed and controlled. It would be insane to claim that, actually, the 6yo using social media every waking minute of the day is totally fine because they are actually having a good experience.

But of course, this is all new legal ground. The case wasn’t supposed to get this far to begin with.

Historically, the big social media companies would avoid liability in these kinds of cases because of Section 230, the law that exempts digital platforms from liability caused by user-generated content. The argument goes something like this: the harms created on these platforms are entirely driven by the content posted by other people; it’s really sad that people become depressed when they see content posted by other people, and the platforms are responsible for monitoring and removing content that is actually illegal to post; but the platforms cannot be held liable when someone becomes depressed because of legal content posted by other people.

This is an extremely compelling argument. There are lots of sad things that happen, and it would be ridiculous to penalize Meta or Google for, e.g., reporting on wars or famines or deaths or whatever. I’ve often said that Meta in particular catches a lot of flak for problems that are really just human problems. People are messy, and complicated, and when you put a billion of them on the same platform you get an exponential explosion of ways things can go wrong. So in general, I think Section 230 is an incredible law, a great example of smart and forward thinking legislators creating the scaffolding for decades of technological, economic, and social progress.3 Judges and juries tend to agree, and have consistently ruled in favor of the big Internet platforms.

Still. There’s clearly a blurry line between “unopinionated information pipe” and “highly editorialized opinion platform.” The gap lies in product design — the millions of choices made that makes Twitter different from Facebook, Facebook different from Tiktok, and Tiktok different from YouTube.

It’s not hard to construct a hypothetical example where the product design is itself clearly harmful. For example, imagine you had a social media company with the ticker MEAT (no relation to any other company).

And imagine that 0.001% of posts on MEAT are really terrible — ai slop racist incitement to violence, that sorta thing. You might rightfully raise an eyebrow if MEAT exclusively showed those 0.001% of posts to everyone browsing their ‘For You’ page. You might raise two eyebrows if the company simply washed their hands of the matter, arguing that because all the content is technically user generated, they have no liability — even though documents show that the code literally has a line that says

if (post == unsettling) visibility = high

It always seemed silly to me to argue that the algorithms that power personal feeds and addictive dark patterns fall under “user generated content”. Legally it’s obviously a grey area, as are many things in the world of software. But just intuitively, obviously the design of a product is the output of a company! What else are all the patents for?

This is, essentially, what the defense argued too. They avoided content. They avoided user behavior. They avoided moderation. Instead, they stayed laser focused on product decisions and the internal discussions and documents that drove those product decisions. And they were able to show liability. Again, from NPR:

The verdict validated the plaintiff’s lawyers’ approach of shifting the legal target; instead of focusing on the content people see on social media, the case put the spotlight on how social media services were designed. Meta’s apps, including Instagram, and Google’s YouTube, the jury concluded, were deliberately built to be addictive and the companies’ executives knew this and failed to protect their youngest users.

Jury trials are famously swingy, so it’s hard to say whether the legal reasoning will stick. But jury trials are great barometers for how the public feels about a topic, and right now, I think the public vibes are rancid. It is conventional wisdom that many companies do try to hack our brains. Everyone in the valley is aware of engagement metrics. Everyone knows that optimizing for engagement works, even when doing so is harmful. Personally I’ve often felt like my phone is channeling some kind of demonic entity. When I have kids they are going to stay as far from phones as possible for as long as possible.4 Most people I know feel the same way. Just about everyone I know is taking steps to try and mitigate their screentime, and failing miserably.

So in the wake of this ruling, many people are cheering that this is finally a way to discourage the addictive patterns that have become endemic to digital life. Including, as it turns out, the jury itself!

Another juror, who gave her name only as Victoria, acknowledged that the jury wanted to send a message to the companies. “We wanted them to feel it,” she said. “We wanted them to realize this was unacceptable.”

But I think it is too easy to blame the big social media giants. Big fines might feel good,5 but they are a hammer when what we need is a scalpel — literally and figuratively the wrong tool for the job.

Imo, all of these companies are reacting to incentives. If Meta puts down its guns and makes their algorithms less effective, they lose users and revenue to Tiktok. If Tiktok puts down its guns and makes their algorithms less effective, they lose users and revenue to Twitter. And so on. This is the kind of coordination problem that government regulation is designed to solve. I think the real indictment lies with lawmakers who have been extremely slow and unwilling to take meaningful steps on these issues.

I have long advocated for a ‘disabled by default’ policy — algorithmic personalization should be disabled unless a user manually enables it.6 I like this approach because it increases user choice, giving more control back to the consumer to set guardrails on their own usage without outright paternalistic bans. I think only the most callous lobbyist would be against this proposal.

But such a solution still requires some kind of law, because again, no one is going to do this unilaterally. I worry that if we keep trying to use the judiciary as a solution here, we will inevitably break Section 230 — either by completely gutting it or expanding it too far.

Right now, Meta and Google have stated that they intend to appeal the ruling. I think this is the obvious thing to do. But I also think this is a missed opportunity stemming from a lack of creativity. In an alternate world, Meta and Google could pledge to do better and then work with lawmakers directly to craft laws that deescalate the attention war. That is a win for everyone — the companies don’t have to spend countless billions on a zero sum attention game, and consumers get their lives back. It also is likely a better outcome for the big tech platforms. Like I said above, the public is angry and frustrated. If the tech companies continue fighting tooth and nail, the end result is not that they will avoid regulation. Rather, like the tobacco companies before them, they will simply not be given a seat at the table.

Other things:

• Delve was removed from the YC website. Delve was the main story of last week’s Tech Things, where we discussed the scandal and the fall out. I mentioned previously that the company was defacto dead regardless of whether it did or didn’t commit the fraud it is accused of committing. YC’s distancing all but confirms that.

• OpenAI acquired TBPN. TBPN is a hilarious concept, a sports podcast for the tech world. It works because they take themselves super seriously, but also because it was obviously a joke. This was an especially interesting acquisition because TBPN is basically only big on Twitter. But that’s also where the entire tech world (unfortunately) seems to be. Generally these kinds of acquisitions don’t have great results for the acquired brand — it’s hard to keep up the same kind of edgy irreverant humor when big money gets involved. Curious to see what happens.

• The Claude Code leak has been all over the news and I don’t have much interesting to say, except that I am very concerned about what this all means for open source. People took the CC leak and had an AI automatically clean-room the code. The argument is that this effectively removes the license. Sure, this feels fine when we’re looking at code from a multi-billion dollar company. But the same logic applies to any other open source library out there. If you can effectively delicense anything with AI, open source licenses as a regime will basically cease to exist. May write a more full post on this soon.

• Everyone who founded xAI left. Probably does not bode well for grok? Setting aside my personal opinions of Elon, I really can’t tell whether he is still making good business decisions. I mean, sure, 10 years ago it was uncontroversial. Now? He has so much money that he’s about as insulated from the market as an engineer working deep in some middle management layer at a FAANG for 15 years. That said, even though I have been very bearish on Tesla self driving, their new models are actually good. It seems throwing enough compute after the problem is enough to overcome (some of) the limitations of not having lidar. So maybe this bet will also pay off? But then I’ve also been bearish on Twitter, and that clearly was correct? For better or worse, the man has created a little universe entirely to himself. A bet on any of those companies is really just a bet on the one guy.

• We’re going to the moon!

I like giving presentations. I hate making slide decks. I spend way too much time making slide decks. Most of that time isn't actually that useful in terms of conveying information. It's bs tasks like fiddling with positioning, aligning text, creating figures. Or it's researching and pulling quotes or summarizing things that I already know but don't have exactly ready for a deck. These things are necessary to have a good presentation -- I've seen many a research PhD slide deck to know otherwise -- but man all these little details bloat out to like 10 hours per deck.

We eventually figured out a hack to use coding agents to bring our average slide deck creation time to ~25 minutes of active work. The finished output is generally more aesthetic, more informative, and more accurate than anything we could do ourselves by hand. Certainly not in the same timeframe.

If you want to skip to just trying this yourself, download the admin skillset (or the creating-slides and record-transcribe skills). In a single command:

npx nori-skillsets install admin  # then just ask claude to make you a slide deck

A few key insights:

• A slide deck as a finished product is separable from the software used to create it. For decades, the only way to make a slide deck is to use a tool like PowerPoint, so it was easy to just mistakenly conflate the two.

• Slides have two modes — a ‘editing’ mode that depends on an internal structured representation to manipulate visual elements; and a ‘presenting’ mode that rasterizes the structure to essentially produce a set of images. There is no strict requirement that the ‘editing’ mode uses a particular data structure.

• Coding agents are terrible at parsing visual media but are fantastic with HTML, which essentially constructs arbitrary visual media using structure.

• Edits to slide decks can generally be done in batch.

With that in mind, here is how we constructed the deck for our board meeting last week.

Give your agent access to relevant context. We maintain an instance of Nori (our coding agent harness) that has access to emails, Slack MCP, design docs, code, and of course, previous board decks.

Tell the agent to construct an HTML page that mimics a slide deck. Emphasize that there should not be complex dependencies, it should really be a single page bare html/css. It helps to provide an example.

Review the deck with audio recording on. My team got in a room and turned on a CLI voice recorder (sox, in our case). We went through the deck and just chatted about what we liked, what we didn’t. Google Meets voice recording will also serve fine here.

Have the agent edit based on the transcript. We used whisper, Google Meets will just provide a transcript, etc. Fed the transcript back to Nori, which went and updated the HTML.

Rasterize. Ask the agent to turn the HTML page to a pdf. It will generally do this without having to download anything else, using chrome in headless mode and the --print-to-pdf command.

The majority of the time was spent reviewing the deck, which we would have done anyway. The agent created figures, positioned everything correctly, used our branding, and even pulled out specific quotes and customer names.

I obviously can’t share the board deck, but I did want to share an example. I created a deck for our open source agent TUI. I did not go through any editing stage for this. These images are basically just slides from the raw deck. It took about 5 minutes to create, and virtually 0 thinking time actually spent.

The full PDF of the deck is here.

Misc other thoughts:

• I’m basically convinced that HTML will be / ought to be the lingua franca for all things visual with coding agents. We’ve started using static HTML sites for everything because it is so easy to just ask an agent to, for e.g., throw up a dashboard of our ec2 instances with a static HTML site. Faster than opening chrome, logging into aws, clicking through the EC2 panel, and trying to parse it. And because the static dashboard that the agent makes is fully customizable, we can often get a better user experience too.

• I’m not sure I would have come up with this flow for generating slide decks myself. My (non-technical) strategic advisor came up with this flow. More generally, I think the folks who are non-technical-technical people are going to innovate a lot, empowered by the agents. This is different than the usual thing that happens, where the non-technical people are years behind the techies.

• Gamma is a startup aimed specifically at taking down powerpoint. They are doing $100m ARR and have a $2.1b valuation. Is any of that sustainable, given the above? How soon before the admins and corporate strategy folks realize that the agents can just do this automatically, and at way higher quality?

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here. Learn more about Nori at https://noriagentic.com/

It seems like the tech world decided to take a break from AI for a month and instead have all the big stories be about fraud. Interesting pivot, if you ask me. Let’s delve into it.

Delve

Traditionally, companies do not like regulation. There are obvious capitalist reasons for this. Regulation generally means that you have to spend money on things like paperwork and auditors and compliance teams and so on, which in turn means either the customer is getting charged more or the capitalist is making less money. But there are also less obvious but equally salient human reasons for this. Regulation is boring. If you’re a founder and you want to go out and change the world, you probably don’t want to sit around doing paperwork proving that your cancer-curing ai-powered glp-1 peptide minimizes harm to the Alaskan red-billed beaver population of Sonoma, or whatever.

Sometimes someone will propose that we get rid of all regulation. And then a river will catch fire a dozen times or someone will start a company called Enron or [everything with sports gambling] and then we go ‘ok maybe we do need regulation after all.’

Still. Even when we recognize some regulation is good, there is an inherent tension here. Regulation is a bit adversarial, because companies don’t really want to do it and because it is inefficient and costly to enforce.

The beauty of capitalism is that any time someone does not want to do something, someone else can make a bunch of money doing that thing.

A founder might go, ‘hey, wow, there are a lot of other companies that hate dealing with regulation. What if I came in and made that easier?’ And then that founder would start a company and would rake in tons of cash.

And a different founder might go, ‘hey, wow, all these audits are really expensive and could be streamlined. What if I came in and made that easier?’ And then that founder would start a company and would rake in tons of cash.

And then a third founder might go, ‘hey wow, there are a lot of people that hate dealing with regulation generally, what if I just helped companies do the audit thing and also did the audits myself?’ And then that founder would start a company, and rake in tons of cash, and then when people realized what was happening, that founder would go to jail. Obviously, obviously, you cannot be both the auditor and the auditee. It’s a massive conflict of interest!

Anyway, Delve. From TechCrunch:

An anonymous Substack post published this week accuses compliance startup Delve of “falsely” convincing “hundreds of customers they were compliant” with privacy and security regulations, potentially exposing those customers to “criminal liability under HIPAA and hefty fines under GDPR.”

Delve is a Y Combinator-backed startup that last year announced raising a $32 million Series A at a $300 million valuation.

The Substack post is credited to “DeepDelver,” who described themselves as working at a (now former) Delve client. In response to emailed questions from TechCrunch, DeepDelver said that they and their collaborators “chose to remain anonymous out of fear for retaliation by Delve.”

[DeepDelver claimed] Delve “achieves its claim of being the fastest platform by producing fake evidence, generating auditor conclusions on behalf of certification mills that rubber stamp reports, and skipping major framework requirements while telling clients they have achieved 100% compliance.”

DeepDelver also claimed that virtually all of Delve’s clients seem to have gone through two audit firms, Accorp and Gradient, which they described as “part of the same operation,” one that operates primarily in India, with only a nominal presence in the United States.

Delve responded to the accusations by saying it does not issue compliance reports at all. Instead, it’s an “automation platform” that ingests information about compliance, then provides auditors with access to that information.

“Final reports and opinions are issued solely by independent, licensed auditors, not Delve,” the company said.

In response to the accusation that it’s providing customers with “fake evidence,” Delve countered that it’s simply offering “templates to help teams document their processes in accordance with compliance requirements, as do other compliance platforms.”

“Draft templates are not the same as ‘pre-filled evidence,’” the company said.

Well, it kind of depends on what you mean by template, doesn’t it! If Delve is sending out, like, empty Google Forms, sure whatever. It’s a bit suspicious, Delve probably shouldn’t be even doing that because of the optics, but fine. But if Delve is sending out, like, finished Docusigns…that’s kind of a different thing altogether. There’s an ocean between ‘Google Form’ and ‘Docusign’, even if they are both ‘templates’ in a strict sense.

The claim is that Delve is doing the latter.

Delve’s model inverts this structure. By generating auditor conclusions, test procedures, and final reports before any independent review occurs, Delve places itself in the role of both implementer and examiner. This is not a technicality. It is a structural fraud that invalidates the entire attestation.

I’m not a compliance guy and I have no idea where this all falls on the axis of ‘totally fine and normal’ to ‘kinda shady’ to ‘jail time’. The Substack post is pretty in depth, feel free to take a look. But to be honest, it almost does not matter. Even if Delve didn’t commit fraud, the company is basically done. A compliance company in the center of a massive fraud scandal, caused by a leaked database due to poor compliance standards? Using Delve for compliance is practically inviting a malpractice lawsuit of some kind.

A few other misc thoughts.

30u30: The funniest thing to come out of the Delve saga is the realization that both Delve founders are Forbes 30 under 30 recipients. When I was growing up, 30u30 was a really impressive badge of honor. But the last ~10 years have not been kind to Forbes. 30u30 is a social club that now includes infamous luminaries like Sam Bankman-Fried (FTX) and Elizabeth Holmes (Theranos); Claude found fourteen prior examples of 30u30 winners who had then either been indicted or gone to jail for fraud. There aren’t that many 30u30 founders! The base rate of fraud has to be way higher than if you were to just random sample the population. At this point, the meme is that the 30u30 nominations are pay-for-play,1 that the only people who get the award are those who aggressively game the system, and that having 30u30 on a resume is a massive red flag possibly even signaling poor character. At least one friend of mine was offered a 30u30 spot and declined the award for that reason. Now there are two more data points.

YC: Delve hit the YC brand too. Delve is a YC company, part of the Winter ‘24 batch. YC is the most famous accelerator in the Bay, possibly the world. Historically it built its brand on:

• Selecting for fantastic talent

• Getting teams in shape to get more funding

• Having a track record of great exits.

Notice the one-two punch. YC appealed to future-round investors because it found great talent, and to talented founders because it could almost-guarantee future round investment. Not a bad place to be! But that brand value has steadily decreased as the company continued to expand its batch size while simultaneously (necessarily?) reducing selectivity. YC’s first batch, in 2005, was only 8 companies. Around 2012, it was ~40-80 per batch. Today, it’s over 150 per batch, and there were over 600 companies that graduated YC in 2025. That means less time per founder, more jockeying for attention at demo day, and, of course, a dramatically increased chance of fraud. Anecdotally, the common wisdom seems to be “join YC for distribution to companies in your batch, otherwise its mixed value,” and there seem to be more accelerators popping up that are trying to compete for the spot that YC previously occupied. All this to say, the Delve thing came at a really bad time for YC. The last thing YC needs is to fend off their previous (rather public) support for Delve. I suspect their PR department is unhappy right now.

AI: As with all things Bay Area ‘26, we must mention AI. One interesting note about the Delve story is how much their marketing leans on AI as their unique edge. Like, this is their home page:

“Delve AI agents eliminate compliance busywork,” they claim. If you squint, the pitch sorta makes sense. Compliance sucks because there’s a lot of manual-but-kinda-rote tasks that you have to do. AI agents are good at doing manual-but-kinda-rote tasks. The syllogism writes itself. The hard part though is, like, actually getting the AI agents to eliminate busywork. It’s a great pitch if the product was real, still gotta make the product real though.

There are elements of the Delve story that remind me of crypto mania a few years ago. With crypto, no one really understood how it worked. Arguably, that was the whole point of crypto. But that meant that you could more or less claim that it did anything, and it was hypey enough that people would fund it. Crypto solves centralization! Crypto will lead to new forms of corporate governance! Crypto will become the backbone of art, no wait, real estate, no wait, medical records, no wait, identity of a person. You could really just say anything! People who knew how crypto worked and who didn’t say insane things would get outfunded by the shameless folks who said whatever investors wanted to hear, regardless of feasibility.

Well, AI is hype now, and AI is technically complicated in a way that few people really understand, so AI is the new crypto. Just create a webpage that claims you can solve perpetual motion, slap some ‘powered by AI’ language on there, and watch the dollars roll in. Of course, AI has a much more obvious value-add than crypto ever really did. But that actually makes it harder to discern which companies are real and which ones are full of it. In some sense, the Delve story is just an old-fashioned memecoin rugpull.

Also, something ironic about naming the company after a word that is commonly used to identify low quality AI-generated slop.

DeepDelver: Speaking of funny names, DeepDelver is great. Really evokes, like, the Watergate scandal. 10/10, no notes.

Polymarket

Here’s an old story, tell me if you’ve heard it already. Back when the Brits were running Delhi, they had a small issue with cobras running amok. You’d step outside and BAM, cobra. It was hard. So to fix this, the colonial government decided to start a cobra bounty program. Folks would get paid for every cobra they killed and brought in. Within the system of the colonial government, this is an ideal solution. You create an essentially-infinite reward structure that effectively deputizes the entire country to act as animal control. The best cobra catchers get paid the most, and the cobra problem clears out pretty quick.

The problem, of course, is that optimizers don’t give a damn about your system, and humans are fantastic optimizers. After all of the low-hanging fruit cobras were picked, folks started breeding cobras for the express purpose of killing them and turning them in for cash. The amount of effort to break the system was less than the amount of effort to do things the ‘right’ way. Eventually the colonial government got wise to what was happening and shut down the program. All the breeders basically just let the cobras go, and overnight the cobra problem got even worse than when they started.

The broader lesson here is that any system subject to optimization pressure has an efficiency ceiling set by the difficulty of breaking out of the system itself. The walls and borders of the system act as a leash on the optimization process, and if they are too weak you won’t actually get much benefit out of the whole thing. Or, more broadly, “when a measure becomes a target, it ceases to be a good measure.”

Here’s a new story, tell me if you’ve heard it already.

On Tuesday, March 10, a massive explosion shook the city of Beit Shemesh, just outside Jerusalem, in yet another Iranian ballistic missile attack during the ongoing war.

Rescue services scrambled to the scene in search of possible casualties, though as it turned out, the projectile had struck a forested area just outside the city, around 500 meters from homes.

On The Times of Israel’s liveblog that day, I reported that the missile had hit an open area and no injuries were caused, citing the rescue services, as well as footage that emerged showing the massive explosion caused by the missile’s warhead.

Later Tuesday, I received an unusual email, in Hebrew, from someone named Aviv.

“Regarding your Times of Israel report that described today’s launch as an ‘impact’ — Beit Shemesh Municipality and MDA (Magen David Adom) later corrected their reports to clarify that what fell was an interceptor fragment, not a full missile,” he claimed…

The event that these people had bet on was “Iran strikes Israel on…?” More than 14 million dollars had been wagered on March 10.

The rules of the bet state: “This market will resolve to ‘Yes’ if Iran initiates a drone, missile, or air strike on Israel’s soil on the listed date in Israel Time (GMT+2). Otherwise, this market will resolve to ‘No’.”

However, there is a clause: “Missiles or drones that are intercepted… will not be sufficient for a ‘Yes’ resolution, regardless of whether they land on Israeli territory or cause damage.” …

Shortly after midnight between Saturday and Sunday, I started to receive threatening messages in Hebrew on WhatsApp from someone called Haim.

“You have exactly half an hour to correct your attempt at influence,” he wrote.

“Despite the fact that you received countless inquiries — you insist on leaving it that way.”

“If you do not correct this by 01:00 Israel time today, March 15, you are bringing upon yourself damage you have never imagined you would suffer,” he threatened, in a very lengthy message.,,

Haim also referred, with specific details, to my ostensible home neighborhood, my parents, and family.

“And as far as I know, there are also some people who don’t really care about the law, and you’re going to make them lose about 50 times what you’ll ever make.”

“86 minutes left. You are the only one responsible for your life.”

In regular markets, the price of an asset carries a lot of information about the future. For example, if oil futures are trading at $60 per barrel, you can infer that smart people think that the war in Iran may end soon, and if it is trading at $200 per barrel, you can infer that smart people think that the war in Iran won’t end soon. But this isn’t, like, the primary purpose of markets. When you buy oil at a certain price, you’re actually literally buying a claim on actual literal oil that is sitting somewhere. The price signalling thing is just a useful secondary byproduct of how markets are structured.

The core idea behind prediction markets is that you can use the decentralized brain of a public market to extract valuable signals about things. There is no asset, no underlying security — the whole point is to incentivize smart people to work on predicting the future (and only predicting the future) by getting paid for being right. Within the system, this works great. People who are right a lot get paid a lot and everyone is encouraged to be more right and eventually we can perfectly predict the future. The problem is that being right consistently is really hard. You know what isn’t hard? Hitting a journalist with a wrench.

Prediction markets have a lot of potential, but this is not the first time I’ve written about how they need more regulation. Right now, it is far too easy to break the system. I have many thoughts on laws that we can use to leash the markets (there’s at least some of this being proposed by lawmakers, see below), but the relevant one for this story is a no brainer: we need official resolution criteria that aren’t set by the random person who created the market! If you have a market that resolves only if “The New York Times reports that pigs can fly,” you may think you are evaluating whether swine are capable of taking to the air. But actually, any signal you uncover is something closer to “The New York Times takes bribes to report on ridiculous topics,” because it is easier to bribe the NYT than it is to have porcine aviation.2

Uncontrolled optimization is a theme that I want to keep returning to, because I think the Delve story also has roots in this kind of uncontrolled optimization for money. There, it turned out that lying about compliance is way easier than actually being compliant, so the optimizers (founders) decided to just lie. Same motivating principle as breeding cobras or harassing journalists.

Super Micro Computer

Supply and demand is econ 101. If there’s a lot of demand for something, and not a lot of supply, then the few suppliers get very rich. For the last few decades, we’ve mostly brought down barriers on who can supply and who can demand. You can be an Australian ordering an authentic New York bagel, and that’s just fine. You’re part of a global pool of demand for bagels and a global pool of supply of bagels, and the market does not care about the number of oceans and borders in between.

Except for a few small exceptions, of course.

If you’re an Iranian you can’t buy an authentic New York bagel, because the US government prohibits it. The US really cares about not giving the Iranians money or supplies — even before the current war — and as a result makes it really hard for anyone in Iran to buy anything at all, much less a bagel. The demand and supply are artificially constrained, which in turn means that if you are Iranian and you really really want a New York bagel, you need to pay up. Big time.

More generally, economic sanctions and export prohibitions don’t ever fully kill a market. Rather, they artificially increase the effective price of a good, making it so that the market clears way less frequently. Ideally, in the eyes of the one applying the sanctions, the market clears so infrequently as to basically be zero. No one would spend a billion dollars on a bagel, so it’s pretty easy to fully shut down the bagel black market with just economic pressure.

But people would spend a billion dollars on things like weapons, or rare metals, or, increasingly, computer chips.

Imagine you’re an executive of a computer chip company. Your chips cost $100 to make, and you normally sell them for $110, and you make a nice $10 profit. And then along comes a sanctioned buyer, and they say “hey, I’ll give you $100000 for one of your chips, wouldn’t you like $100000?” On the one hand, this is an illegal trade, and if you accept the $100000 you may go to jail. On the other hand, you make a nice $99900 profit, which is 9990x more than you would have made otherwise. And, like, how bad is jail, really? Apparently jails for rich people are kind of nice, and you’d get to pay off your kid’s college, and you’d get a nice car…the point is, you can see how this may be a tempting offer.

Too tempting for Super Micro Computer (SCMI):

The U.S. government has been trying to figure out how high-powered chips have reached China without authorization, as American artificial intelligence companies such as Anthropic and OpenAI face challenges from DeepSeek and other Chinese rivals.

In an indictment unsealed Thursday, the U.S. government alleged that Yih-Shyan “Wally” Liaw, Ruei-Tsan “Steven” Chang and Ting-Wei “Willy” Sun worked together to violate the Export Control Reform Act.

The server company’s products containing Nvidia chips “are subject to strict U.S. export controls barring their sale to China without a license,” the plaintiff said in the indictment. “Those controls are in place to protect U.S. national security and foreign policy interests, among other things.”

The efforts have yielded around $2.5 billion in sales for the server maker since 2024, with servers sold for $510 million between late April 2025 and mid-May 2025 going to the Southeast Asian company and on to China, the indictment said. The plaintiff said the server maker had no U.S. Commerce Department license to export servers featuring Nvidia GPUs to China.

“Roughly how many you can take by January? Feb? March? April?” Liaw wrote in a text message to an executive at the Southeast Asian company. “Just roughly forecast will be fine ... Then we can propose to [Nvidia] with the way they can accept ... This is the only way to have [Nvidia] to promise the B200 allocation so far as I know.”

When a broker who had bought Nvidia-powered servers from the Southeast Asian company sent Liaw a text message containing a link to an announcement about Chinese nationals being arrested for smuggling AI chips into China, Liaw allegedly responded with sobbing emojis.

Seems like Wally was breaking the Liaw.

We don’t know exactly how much the spread was between what the Chinese companies were paying and the normal server prices. The $2.5b is only what showed up on SCMI’s balance sheet. Claude thinks that the spread was about 50%, meaning that Wally and his conspirators captured ~$1.25b. That’s a lot!

I think that the most silly part of this story is that this isn’t even the first time SCMI has done this exact trade. Back in 2006, SCMI got in trouble for selling servers to Iran. And in 2016, SCMI got in trouble for selling servers to China. And then, in 2022, SCMI got in trouble for selling servers to Russia. And now, in 2026, SCMI is in trouble for selling servers to China, again. At this point, they basically specialize in selling to sanctioned countries! That’s their whole brand! Which, honestly, isn’t really the worst market niche, purely in business terms (not business advice). If you’re a sanctioned country, and you want chips, you know exactly who to go to. Extremely strong product market fit.

Other things

• Anthropic was granted a preliminary injunction on the supply chain risk designation. The judge in the case was really not happy with the US Government, stating bluntly that the “designation of Anthropic as a “supply chain risk” is likely both contrary to law and arbitrary and capricious…Nothing in the governing statute supports the Orwellian notion that an American company may be branded a potential adversary and saboteur of the U.S. for expressing disagreement with the government.” Generally if a judge uses the word ‘orwellian’ to refer to your side of the case, you should not be bullish on the odds of success.

• People are taking prediction market insider trading more seriously. “Reps. Nikki Budzinski (D-Ill.) and Adrian Smith (R-Neb.) will introduce the Preventing Real-time Exploitation and Deceptive Insider Congressional Trading Act, or PREDICT Act, Tuesday to ban members of Congress from participating in prediction markets related to political events or policy decisions. The ban would also extend to dependents and spouses of lawmakers, senior congressional staff, political appointees, the president, vice president and all senior executive branch employees, including special government employees.” In favor of this. Also, how much time do you think lawmakers spend just trying to come up with clever acronyms?

• NY proposes removing qualified small business tax exemption. Normally, if you start a small business and run it for a certain number of years, and then sell the business, you do not get taxed as much on the sale. This proposal would make it so that you would be taxed at the state level. Obviously I’m talking my own book here a bit, but I think this is bad. Startups are a center of innovation. Founders are actually job creators, they bring billions of dollars to the state, and in NYC in particular have been creating a economic (and political!) counterbalance to the bay. They are also highly mobile and very sensitive to risks to their exits, because startups are such a massively risky endeavor to begin with. Hoping they don’t pass this. If you know other founders in NYC, let them know.

This image is the TLDR.

Dev tools are, broadly, any piece of software that is used to make it easier to create software. IDEs are the canonical example. A good IDE exists to make the process of writing code easier.1 But there are others. Version control, code review tools, package managers, debuggers, testing, logging and observability, project management suites, etc. etc. If you expand your definition a bit, you could include cloud platforms, web containers, and more.

Our understanding of dev tools is grounded in ~60 years of code being expensive. Originally, code was expensive because the machines were expensive. Think: big super computer sitting in MIT’s basement, so pricey that you had to literally reserve your time on the thing like it was a NYC Resy top pick. Later, code was expensive because the people writing the code were expensive. There’s an old (probably apocryphal) story about how Larry and Sergei hated meetings so much, they toyed around with having a cost-of-meeting counter that would sum the per-minute salaries of all of the attendees and slowly tick up the cost as the meeting went on, but even a ten minute meeting had an obscene ticker cost and ended up discouraging all meetings.

But code is cheap now.

Increasingly people are just not writing code by hand at all. They use coding agents. The cost of a single line of code, previously denominated in tens of dollars, is now fractions of a cent. So, naturally, everything we know about software engineering is undergoing a seismic shift as the tools reorient from ‘person first’ to ‘agent first’. As evidence, see the CLI-ification of everything.

My team has been thinking about AI dev tooling since Jan 2025. With the rise of coding agents, we are seeing increasing consolidation around six areas of dev tools. I think that any team that is using coding agents seriously will need to either build their own solutions in each category, or buy products in each category. Let’s dive in.

Interaction

This is the UX layer of the coding agent, equivalent to the IDE of the previous generation. The thing the developer opens every day in order to interact with and guide a fleet of agents that write the actual code.

IDEs always had a fundamental limit: they had to actually show the code on screen. That meant that there were only so many ways to create an IDE. Individual IDEs varied in terms of integrations and interaction customisability, but the actual UX basically always involved a big text editor that takes up 80-90% of screen space. That form factor also means you need at least 10in of screen space. Even though there were experimental mobile IDEs, they never really took off because they never were ergonomic.

In a world where you do not actually manually write code, you do not have to see that code most of the time. That means an explosion of UX options, and a fragmentation of the market as individual people gravitate towards form factors that they like. We’re already seeing a few different UX patterns:

1. Terminal UI: Claude Code, Codex, Gemini CLI, Nori-CLI, Crush, OpenCode…

2. Multiplexers: Humanlayer, Antigravity,

3. IDE: Cursor, Windsurf, avante, Zed, various plugins to VSCode

4. Background / Slack: Devin, Claude Web, Stripe Minions (internal), Ramp Inspect (internal)

I don’t think anyone has cracked mobile coding agent UX yet, but it feels inevitable — you should be able to kick off jobs from your phone, the same way you can message a co-worker to tell them to work on something. OpenClaw feels like a step in this direction, though from a UX perspective it is relying on a standard chat interface.

It’s not clear that a good UX is, on its own, a sustainable business. VSCode has become dominant in the IDE world, but it is a free product sustained by Microsoft.

Still, developers will pay for tools that feel good to use, and business will pay for tools that are integrated to the larger corporate environment / process. You could imagine a coding agent UX for doctors (for eg) that just makes it really easy to hook into patient data systems like Epic or MyChart. The usual marketing playbook applies: if you are the obvious choice for <job category> then there’s money to be made.

Context

LLMs are zero shot learners. They learn a lot about the world from their training data, but the most important thing they learn is how to extrapolate from the patterns they are given. That means it is really important to manage the context your agent is running on. In my experience, many people who have bad experiences with coding agents are frustrated that the agent does not intuitively understand things that are not obvious. At Google, we had projects that were named all sorts of things. “We have to use red ant to get data from arachne so we can train starburst models” is entirely coherent at Google. But out of the box, the AI will never be able to properly parse that sentence.

I’m told that “Institutional memory” has become a big thing in the enterprise sales orgs of OpenAI and Anthropic. How do we get the right company information into an agent, especially for larger companies when even the search index for the total possible document space is way larger than what a single model can reasonably hold in context?

There is a lot of opportunity in this space because ‘context’ is so generic. We’ve seen a few strategies.

Give the model access to company data directly. MCP, Bash CLI integrations, and custom tool calling all fall into this category. Instead of changing the data or the data warehouse, simply give the model access to the underlying query system the same way a human may have access. MCP in particular is having a bit of a resurgence after mostly falling off the map a few months prior due to the popularity of SKILL files. Most of the companies operating in this space are aggregators — they create and maintain a library of integrations and create an easy drop in to access all of them. Credentials and authentication are still an issue though; more on that later. Aggregators include Composio, Manufact, Pica. If you expand the aperture a bit, you get products like fireflies (call transcripts to agent context) or sentience (consumer social to agent context).

Modify and compress the data to fit the model. SKILLs, AGENTS files, various forms of documentation all fit into this category. Certain kinds of organizational knowledge do not fit well into the traditional query format, because they represent processes instead of data. For example, think about the checklist of steps that needs to be done to do a security audit. Where would you store that? It doesn’t really make sense to put that in a postgres DB and tell the model “call this data when you feel like it.” If you’re doing a security audit, you need the checklist every time! The companies and projects in this space often look like package managers, or are literally just bundles of configs. See: Context7, Nori Skillsets, skills.sh, Enact, skillsmp. Increasingly, companies are also just releasing their own individual SKILLs bundles, e.g. this set of skills from astronomer.

I think building startups in the context space is very difficult, because it reaches asymptotic saturation within a company very quickly. Once a company has done the initial setup to get their integrations in place, why bother paying a subscription? The companies playing middle men may end up continuing to extract value, but I suspect that they will be squeezed by teams realizing that they can build their own integrations relatively easily.2 More generally, I’m somewhat skeptical of companies that follow the 5tran / zapier model. Those companies thrive in a world where code is expensive. Building an integration isn’t hard mechanically, it just takes time, and time is money. But as I keep saying, code is cheap now. So the marginal value of an additional integration point is way lower. My hunch: there will be a few aggregators that win3 and the rest will pivot to something else.

Memorization

The context layer is about getting information into the model. The memorization layer is the inverse — getting information out of the model. Recursive self improvement sounds like scifi, but there is a lot of low hanging fruit infrastructure that can make models better over time through some kind of ‘continuous learning’. This doesn’t take place in the model weight layer. Rather, it happens by constantly manipulating, updating, and compressing the context layer.

The name of the game is transcript analysis. Nearly every tool in this space creates some kind of watcher process that operates in or on transcripts between employees / users and their agents. Sometimes the transcripts are enough. The product is simply a search server over the transcripts, exposed as an MCP or CLI tool to the agent in the context layer. Other times, the product is some kind of additional analysis, some kind of aggregation step that uses graph / vector / metadata analysis to edit the context layer.

The core differentiation here ends up being ease of use and how the update loop is handled. Companies with clever and proprietary update loops will likely do better than those that are just ‘ingest and expose’. For example, at Nori, we do the usual transcript ingest, and then we use those transcripts to augment skills and skillsets instead of just making the transcripts blindly available.

Open Source: beads, Graphiti, A-Mem, SimpleMem, MemOS, OpenMemory, memU, Khoj

Companies: Nori, Entire, Mem0, Letta, Cognee, Zep, Supermemory, Pieces, Augment Code, Fireflies

Infrastructure

In some sense, all of the above is infrastructure, so I want to be a bit precise here. When I say infrastructure, I mean “where, exactly, is the agent running?” There are really only three answers.

1. On the model provider’s infrastructure. When you run Claude or ChatGPT in the web browser, you are running an agent on their inference machines. The agent maintains state, has access to tools, etc. but you have no ability to see or modify any of that. You cannot add tools, you cannot easily modify files. It looks, for all the world, like a chat bot. And that’s how the big model providers want it. This is the non-tech pure-consumer area, and is basically always going to be deeply tied to the model layer.4 If you have multiple conversations, they do not really have any ability to interact with each other (at least, not through the computer).
   

2. On a local machine. The agent runs on your computer like any other application. This is how virtually all of the coding agent cli tools run out of the box. Same with Claude Cowork or Cursor or Antigravity. Generally, the agent itself is pretty lightweight. It’s just a for-loop with some bash. But the agent can run very heavy processes, like builds or test suites. That in turn means you can get interference. Multiple agents running on the same machine can spike memory or block CPU. And of course, they can interact with the same files on the filesystem, leading to all sorts of weirdness. Also, a bit of a pet peeve: you cannot easily run the agent on the go. Since the agent is tied to the machine, you are stuck attached to the machine. Close the laptop, and the music stops. So why do anything this way? The main benefit is that it’s all local. You have full control of the context and all of the authentication. These are nonnegotiable for anyone who is trying to use agents for real developer work (and even more so in most enterprise settings that care about regulations and identity).
   

3. On a stateful remote box. This is generally a hybrid. You get some of the benefits of configuration and more control over the underlying machine, while still having remote sessions that can be accessed from anywhere that you don’t have to personally orchestrate. But there is a lot of variability here. Depending on how the infrastructure is set up (and for whom!) you could have anything from ephemeral isolated machines that can be seamlessly accessed from a bunch of different platforms (e.g. slack) to a big single box that is just treated as a remote dev machine to something that is a very thin wrapper over AWS with no additional features (Fly, Modal). Coding agents can do a lot, but they are not quite at the point where they can build a seamless multi-machine orchestration system. Ramp Inspect and Stripe Minions both follow this pattern for internal use. Nori Sessions follows this pattern, but as an off-the-shelf product. Other tools in this space: Oz, Coder, Ona

The main axis that matters is configurability — how easy is it to get into the machine and change what is on it? On face it seems like local usage should be squeezed out. But in practice, the literally-0 setup time and maximum configurability means there will always be demand for local infrastructure, especially among individuals and small teams. My team has a robust ephemeral cloud offering with nori sessions, and I’ll still pull up my local coding agent during the work day (I’ll just teleport those sessions to the remote when I want to leave my desk).

As with many things in this post-software world, I suspect a lot of value will accrue to compute providers. Building in the infra space is just a classic pickaxe business, and for highly autonomous coding agents they make so much sense. You want to be able to access coding agents from anywhere, so you need to build infra that supports that, and the infra usage is directly proportional to the agent usage.

Testing and Verification

I think historically, testing has been under-appreciated. Most people who write code don’t write tests. That is because they verify the program they write as they go, based on the outputs. The act of writing the code is itself an act of problem solving. And writing tests by hand is as time consuming as just building whatever you need to build; no one wants to double their investment for 0 up front additional gain.

LLMs fundamentally change the calculus around testing. Testing and verification has become one of the most important areas of development for modern LLM-enabled teams. Two reasons for this.

First, LLMs are stateless. They lose all memory from call to call. An LLM could write some code at 2pm and have no memory of it by 3pm. They are constantly seeing a codebase with fresh eyes, which means they are constantly at risk of misunderstanding something and unintentionally breaking some feature.5 Tests act as a backstop for coding agent error. Comprehensive testing is a memory mechanism.

Second, it increasingly seems that many software problems are search problems. If you have enough compute you can run enough LLMs to figure out a solution to just about any test case, or drive down some optimization criteria. For software products, this inverts the usual thinking: instead of building the product, you build test criteria that the product needs to meet, and then let the LLM finish the rest. So the upshot is that you can (must?) spend far more time thinking about product instead of implementation. LLMs have driven the cost of writing tests down in tandem, so building product has essentially become an exercise in thinking about comprehensive specification more than anything else.

As teams begin to fully grasp the implications of coding agents, they will need to invest in emulators, good high quality mocks, and testing infrastructure.

This is both very hard and very expensive. If you have an app that runs on slack, for eg, you are going to want tests that check slack behavior. How do you do this? To be complete, you would have to rebuild a lot of slack infrastructure as a local mock. But that in turn is a pain. What if you got your implementation wrong? What if slack changes out from underneath you? Also, slack has a lot of features with all sorts of complex interactions. Are you going to mock all of them? Are you going to do this for every dependency you have?

For some teams, the answer is yes:

Which leads us to StrongDM’s concept of a Digital Twin Universe—the part of the demo I saw that made the strongest impression on me…

[The Digital Twin Universe is] behavioral clones of the third-party services our software depends on. We built twins of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, replicating their APIs, edge cases, and observable behaviors.

With the DTU, we can validate at volumes and rates far exceeding production limits. We can test failure modes that would be dangerous or impossible against live services. We can run thousands of scenarios per hour without hitting rate limits, triggering abuse detection, or accumulating API costs.

This screenshot of their Slack twin also helps illustrate how the testing process works, showing a stream of simulated Okta users who are about to need access to different simulated systems.

Tools like localstack6 or playwright are invaluable here, and companies like Mercor or Mechanize explicitly sell RL environments.

Of course, the other approach is to just do it live.

Instead of investing in mocks, you just run the real thing. Spin up your actual app and then just try to break it by doing a bunch of things that a user would do. Here, LLMs act as QA operators. You feed an LLM what is essentially a markdown file describing a particular feature spec, and the LLM just makes sure the spec is met. In the limit, you could imagine CI/CD platforms evolving into full suites of agents trying to break every critical feature path. End to end integration tests are hard because they are so varied, and this is very much not a solved problem. Web developers and terminal developers have big advantages here, because it is fairly easy to emulate both of those environments. Companies in this space: Momentic, Functionize, Morph.

I think there will be a lot of demand in this space, but it is also really unclear how to avoid being super fragmented. Maintaining even a single really-high-quality mock environment seems hard? In the StrongDM blog post above, they talk about spending over $1k in tokens per developer per day. Seems like a lot, and I think they are understating how much they are able to clone entire applications from just looking at client libraries. Also, separate from the above, I have a strong suspicion that much of the value in using LLMs as QA is captured by the infrastructure layer that those LLMs are running on. After all, the QA model is basically just ‘feed a markdown file to your LLM’. Does that become a full product? Very unclear.

Identity and Auth

And finally we get to the security layer. I mean, what can I say that hasn’t been said by two dozen posts about how scary OpenClaw is?

The fundamental contradiction of using AI agents for ‘real’ tasks is that the agent is only as good as the amount of context it has and the number of actions it can take. An agent that has 0 tools and no context is barely even an agent. But an agent that has access to everything can seriously screw things up. People rightfully get spooked by the idea of giving agents access to api keys, ssh credentials, or, like, a credit card number. Forget about context injection or malicious prompting, the AI itself is fundamentally untrustworthy! Every time you interact with it you get a totally new session, and who knows if this session is going to behave or going to threaten you?

If you can’t trust the agent, you need something that you can trust, something that will be responsible for keeping the agent in line. As far as I can tell, the only way to do this is to put some kind of gate-keeper in between the agent and the external world. The question is simply where — or, more specifically, how far from the agent.

One place you could put the gate is inside the agent harness tool calls. A tool call is just code, and it works like any other code. So you can have tools that are authenticated, have some kind of permissions, and may even have externally injected secrets. The MCP spec supports this outright as an OAuth2.0 client, though anecdotally it does not seem like many MCP implementations really futz around with the auth side of things. See: Descope, Stych, Scalekit, Credal.

Another place you could put the gate is in front of the critical resources you care about. For example, you could imagine a service that sits in between your db. It perfectly understands the underlying protocol, and you can write up policy that is enforced in code to ensure that agent queries are logged and (where necessary) blocked. For example: Empathic, Formal, ProxySQL

The middle ground is a proxy. The agent thinks it has access to everything. The databases and third party APIs don’t know or care about the caller. In between there is a little service with all the responsibility, that ingests every message from the agent and analyzes the transcript for things that could be going wrong. See: Portkey, Agent Gateway, MCP-Proxy.

I’m most bullish on the last approach from a technical lens because it is generally transparent to both the agent and the underlying sensitive resources. But I also know that the shape of any authentication layer will need to very closely match the industry. Consumer vs enterprise. PII vs PHI vs PCI. Regulatory requirements for paper trails and humans-in-the-loop. And I’ve mostly conflated identity and authorization in this section, but those are potentially two very different things. Agents can run autonomously without having a person behind them, so who do they ‘represent’ from a liability perspective? Products in this area are very much in the ‘figuring it out’ stage, but the ones that do will be able to tap the big capital reserves currently sitting in banks and healthcare.

The through-line here is efficiency. Before, it did not make (as much) sense to think about how computers could identify themselves. There was always a person behind the screen. You don’t need to think about “context”, that’s what the first three days of onboarding new interns is for. You don’t need to think about “memory”, humans do that automatically. And humans do things at human speed.

Now that agents can act autonomously — and faster and more consistently than humans — the burden has shifted from “make humans more efficient” to “make agents more efficient.” Thus the tooling. Every single product described above aims to pull people out of the loop while giving the agent more agency. Some of that is explicit, eg tools and skills. Some implicit, eg better auth means more trust means ability to run agents more freely. But the end goal is the same.

One thing that is currently unclear is whether the large labs will suck all the air out of the dev tooling space. They are clearly trying. Anthropic has Claude Code (interface), skills (context), automatic Claude memory (memory), Claude Web (infrastructure). I’m not aware of any testing or authentication solutions, but any team using agents effectively will run into these problems and concerns, so I wouldn’t be surprised if something in those categories was in the works. Though I did not touch on the model layer above,7 there is leverage in owning the model layer. The big players could price their products to extract all the value, or they could fine tune the models to work best on their own developer tooling.

But right now, it would be profoundly arrogant to assume that any company has a lock on any part of the dev tooling space, or even to say with confidence what the dev tool space will look like in a few months/years. Lower barriers to entry mean an increase in diversity. It is far more likely that the market will fragment based on taste and aesthetic and minor feature convenience, things that are important to small market segments that the big labs simply can’t justify tackling. Let a million flowers bloom, as the saying goes.

Does this mean that everyone will simply build their own tools? After all, that is the best way to get maximum configurability. Where are the software moats?

If you ask me, the build vs buy line has changed, but it hasn’t changed that much. The majority of programmers are still people working unsexy desk job 9-5s at your major bank chain of choice. These people are not going to invest a ton in building their own AI dev tools, any more than they were likely to build their own IDE or Kanban board. Right now, there is no clear idea of what AI dev tools will look like or who the big trusted players are. CTOs and eng teams will fill that gap, because the gap needs to be filled. But when good, convenient, easy to use alternatives come along, I strongly suspect those same teams will offload their maintenance burden to other companies. A health tech CTO (for eg) probably doesn’t want to think about maintaining dev tools. Especially if those dev tools have a bunch of rough product edges that need to be rethought from first principles. The software moat is now very much a product moat. If your product feels good and is easy to use, it will still win out. And as I said before:

Every line of code is a liability, and there is always going to be demand for “you deal with this so I don’t have to.”

Last thought. You may notice that many companies appear several times in this article. As I said, ‘any team using agents effectively will run into these problems and concerns’. Teams that are using AI to build AI dev tools will naturally go out and build other parts of the dev tool landscape! That is 100% what we are doing at Nori. Looking forward a year, I suspect many AI dev tool companies will have suites of solutions, all of which will be considered table stakes for the AI dev tool category.

Agentics is the study of how to use and reason about agents. If you are an expert in coding agents, or interested in learning more about agents, join our community slack. More articles here.

If you are using coding agents and are thinking about building your own remote sessions infrastructure, check out norisessions.com and reach out instead!

You May Also Like:

I’ve been invited to give a lot of talks about coding agents over the last few months. Big companies, small startups, VC, PE. So far all of those talks have been behind closed doors. But this week I was invited to give a talk at the OLAB at NYU Langone! And since this one was relatively public, I was able to finally post the talk itself.

The recording starts like ten minutes into the talk (because, of course, we didn’t think to properly record it at the beginning, whoops). But there are a bunch of nuggets of wisdom in there that I think are key to how we use coding agents at Nori.

If any of this is relevant to you, feel free to reach out. Happy to give a talk to your team if it would be helpful and relevant.

If you’re a tech team feeling the FOMO and struggling to build your AI capabilities, reach out. My team builds remote agent runtimes and company context so you can do what the best engineers at Stripe and Ramp are already doing. Or join our public slack to keep up to date on what’s happening in AI in real time.

The ironic thing about the founder journey is that after all is said and done, most founders will eventually leave the companies they create. Sometimes this is because you had a great exit. You get out of the souped-up car that you’ve worked on, polished, slaved over, and hand the keys to an exec at a mega corp. Sometimes it’s because the company is dead in the water. You turn off the life support and return any cash you have left and lament what could’ve been. And sometimes, most painfully, it’s because you can’t stand your company and your cofounders any longer, and you need to get out (or risk jail time for assault).1

In the span of a month, two close friends left their companies. One had a buyout that couldn’t meet the company pref-stack. The other was pulling his hair out due to co-founder conflict and needed to quit. They’re both still going through it. It’s hard.

When I left my first company, I left on pretty good terms, and I still had a pretty rough time of it. With a year+ of hindsight and a lot of writing in between, I think there are a few things that make leaving a company really really hard for founders.

• Most founders that I have met are ambitious, hyper energetic people. We can’t sit still. I suspect half of us are borderline manic most of the time. That energy is what sustains a founder. But it also makes leaving hard, because leaving means a vacuum. What do you do with the time? How many times can you do the routes at your local climbing gym, how many runs of Ascension 20 Slay the Spire before all of that sitting around starts to get a bit boring? Related: many founders are not exactly in tune with their emotions. Huge competitive edge when slugging through hour 33 of writing code, less good when left alone with your thoughts.

• Leaving a company is like leaving a family. Founders have close personal relationships to the people they hire; the best founders love and live for their teams. Leaving a company feels like a betrayal. “I joined this company because I wanted to work with you, and I put my faith in you, and now you’re leaving?” “How could you not see how hard this has gotten, why did you not do more to help?” Some founders lose their entire social network when they leave their companies, especially in the Bay where social life and work are so closely tied together. If you have a big co-founder fight, are you supposed to just meet up at the next ML Paper Reading Group & Cuddle Pile and chat-slash-cuddle like nothing happened? Founder breakups are more toxic than most actual breakups.

• Founders can’t actually talk about their departure, at least not publicly. Certainly not to the people they are leaving behind. Founders often have non-disparagement clauses tied to their equity. So you can’t share why and how you left with anyone in the industry, unless you do so with a smile on your face. Which, again, is many of your closest friends! Also, many founders are pathological rule breakers and misfits who see their work as part of their identity and self expression (myself included). Restrictions on speech hit especially hard.
  

• Founders wrap so much of their personal identity and self-worth into the success of their company. If the company failed, it’s because I failed. I am a failure. I didn’t work hard enough, or smart enough. That same mentality keeps founders in terrible working conditions.

All of this lends itself to a really common failure mode. Founders instinctively try to control the ambiguity — the moment they leave, they are setting up their next thing. This time I’ll make it. Some founders go further and try to recreate their first company. I’ll do it better this time, I won’t make any of the same mistakes.

Don’t do this.

If you spend months-to-years working on whatevercorp.ai, your entire way of thinking is clouded. You will see everything through the prism of being the founder of whatevercorp.ai, and as a result you will make bad and probably incoherent decisions.

Instead, you have to do something that is basically impossible for a lot of founders: slow down, sit back, and reconnect with yourself.

To the extent that I have advice, here are some notes:

• You need to clear your thinking, let the oil and water separate. The best way to do that is to dramatically reduce the timeframe of your thinking. You’re used to thinking weeks or months or years ahead. Stop that. Bring your timeframe down to one month. Then one week. Then one day. Then five minutes. What do you want to do five minutes from right now? Answer that question honestly, and then go do that thing. For the first month after I left my first company, I had huge plans about starting all sorts of projects. But what I really wanted to do was eat Costco taquitos and play Pikmin. I am so grateful I did the latter, because that eventually led me to what I was naturally curious about (coding agents!).

• Related: avoid the impulse to answer questions about what’s next too early. It’s fine to not have an answer when someone thinks they are being helpful and “a good friend” when they inadvertently ask ‘what are you planning to do next?’ It’s fine to say ‘I don’t know’ and mean it. There are no hard and fast rules, but I think if you have an answer within a month you didn’t spend enough time thinking. You should seriously consider whether you even want to do the startup thing. It’s fine if you don’t!

• Write. Write a lot. Write your founder journey. Don’t use AI. All this serves two purposes: first, writing is thinking, and you need to make time and space to think and process; and second, future you will want that memoir as something to look back on. Writing has this beautiful way of cutting through emotion, because you can look back at what you wrote after a night’s sleep and see in your writing a different version of yourself, and recognize that that version isn’t you now.

• Don’t travel. Stay in your hometown and let yourself be bored. The brain is remarkably efficient. It builds routines — including emotional states — around physical memory. You spent months working on your company. Your apartment / home and your city and of course your office will trigger certain emotions, and you need to rewire. Travelling is great, and can be relaxing, but travel is about creating new routines in new places. You need to give yourself some amount of time going about your usual day so your brain will understand the new normal (alt: just move. This is obviously less immediately attainable for most people, but would have a similar effect!)

• Get advice from other founders who have previously left their companies. This is not always easy to do, because you may not know other founders who have left their companies, or you may not feel like you can approach them. One of the things I don’t like about Bay Area founder culture is that everyone is constantly showing off to everyone else, which means there is little space for vulnerability. Even when people are really going through it, they will grit their teeth and smile and give the pitch for the ten millionth time. So, yea, not easy to find folks who are good to talk about these things with. But if you do have someone in your wheelhouse who has gone through it before and who you can talk to, call them. They will have a lot of detailed things to say, and will likely appreciate the call.

These are the things that were most useful for me. There are also the usual canned-but-useful bits of advice — spend time with friends, pick up new hobbies, get fit, sleep, read, etc. etc. Those are all helpful, but I think far less so than the five things mentioned above.

The last thing I’ll add here is that if you are reading this because you just left your company and it resonates:

• first, I’m sorry that you had to leave under less than ideal conditions;

• second, congratulations, this is basically always the right decision, and I hope you feel a lot of relief;

• third, it gets better, and you will eventually become your previous company’s biggest ally (because, you know, you are likely heavily financially incentivized to do so);

And finally, if you need it, feel free to reach out, happy to chat.

I want to reflect briefly on the surge of nihilistic discourse that seems to increasingly plague the tech community. This particular disease goes by many names — ‘black pill’1, ‘stoicism’2, ‘pragmatism’. But don’t let that confuse you. These are all forms of the same thing.

The core of this mindset is a logical fallacy:

• The speaker has realized that ‘everything is made up’, that ‘there are no rules’, that ‘you can just do things’;

• therefore having values, beliefs, or morals at all is misguided, naive, cringe, and uncool.

A classic example that exemplifies the mindset is something like “but it gets clicks/views/engagement.” Another one: “this is just how the world works.” A third: “it’s not my problem.”

I first noticed an exaggerated version of this mindset when I spoke to a group of young 20yo founders in April of last year. I wrote:

Here was a group of people who were, by every account, living some of the most blessed lives on the planet. They were all healthy. They had decent relationships with their family and friends. They were surrounded by people who were eager to give them the resources to fulfill their wildest dreams. That’s not even an exaggeration. In between complaining about the decay of the city, one of the group mentioned off hand how a VC had bought him an 8sleep mattress and sensory deprivation spa treatments. I looked it up later — the 8sleep that he got was worth $5000 dollars. At the same age I was sleeping on a $100 foam/plastic twin bed.

In spite of their obvious good fortune, talking to them was a bit like talking to a 4chan greentext. They would joke about edgy bullshit — how it was good that 10 year olds were addicted to phones so that they could make money off them, that it was time for democracy to die and be replaced by techno feudalism, that we should renormalize homophobia as a society. It took me a while to realize they were serious, they were laying the irony on so thick. Why on earth were they so cynical? What did they have to complain about, that they were all so eager to “burn it all down”?

Why were they so eager to burn it all down? I never got a satisfying answer. But in the intervening year, I saw this thought pattern metastasize into the wider ecosystem, turbo-charged by the industry’s reliance on Twitter as its platform of choice. And now the nihilism just feels like it is everywhere, from the prevalence of startups getting funding for gambling apps to crypto scams to the naked use of AI to destroy the commons in pursuit of money.

I understand why the nihilism is baseline seductive. There is a lot of truth to the idea that many things in the world are artificial constructs. Founders and early stage employees understand this more deeply than most. It turns out that many company norms and rules exist because some guy just decided that that’s how things would go. I vividly remember when I had to pick my company’s insurance policy. Like, I read through the policies and did my best but at the end of the day, there I was, some guy that just decided that’s how things would go.

If you are in the startup world for long enough, shifting norms — or outright breaking them — becomes a learned muscle. And you start to see those norms everywhere. Sometimes this is helpful. “I know your policy says you can’t give me a refund, but also I know that’s just something you have to tell me, so let’s see if we can work together and give me something partial.” Works almost every time.

The problem, of course, is in the second part of the syllogism.

Bluntly, 2026 isn’t the first year where someone realized that there is no inherent morality to the world. This is a subject that philosophers have debated for literal centuries.

“Meaningless! Meaningless! Everything is meaningless. What do workers gain from their toil? I know that there is nothing better for people than to be happy and to do good while they live.” (Ecclesiastes) “Waste no more time arguing about what a good man should be. Be one.” (Marcus Aurelius)3 “I do not reproach nihilism’s arrival. I believe it is one of the greatest crises, a moment of the deepest self-reflection of humanity. Whether man recovers from it, whether he becomes master of this crisis, is a question of his strength.” (Nietzsche) “One must imagine Sisyphus happy.” (Camus)

The nihilist founder thinks abandoning ethics is the brave, clear-eyed position, and that anyone still following some sense of order is a naive sheep.

That’s cope, used to justify a ‘got mine, fuck yours’ mentality. Opting out is still a choice. Cynicism is still a value system — just a lazy and self-serving one. The only thing that distinguishes humanity as a species is our ability to define values and seek out individual ethics despite an unordered and chaotic world.

And this is hard to do! It is genuinely extremely difficult to build a coherent ethical system. It requires a lot of uncomfortable questions about what you actually believe, even when those beliefs cost you something. Especially when they cost you something.

That does not mean “never compromise.” Compromise is healthy and necessary, both as a founder and, like, as a person. You need to find a medium between ideological purity and complete rejection of ethics.

But as someone who has gone through one founder arc, at the end of every startup journey you are left with yourself and your decisions. And I have never, ever met a nihilist who was fulfilled and happy, even when they have good exits. The nihilism leaves a void that even success cannot fill.

(If they even have good exits! I can't rigorously prove it, but I strongly suspect founders that have real ethical beliefs are on average more likely to be successful at running a business. It's easier to hire, easier to get customers, and of course easier to have conviction about what you're building when it gets hard in the long haul.)

All this to say, if having beliefs makes me uncool, so be it. I’d rather be uncool than be a nihilist.

Final thought: I think there is a lot that more senior members of the tech community can do to stem this self-destructive (and society-destructive) mindset. We should act in accordance with our own beliefs and think more deeply about where we compromise. And we should guide the ever-younger members of the Valley towards healthy founder mindsets and ways of being in the world. If you fall in that latter category — if you’re a founder and are unsure how to be — feel free to reach out. DMs open.

You may also like:

To that end, I am very happy to have Dan Parshall writing on the blog today. Dan is a former physicist and data scientist working on AI policy at the Canary Institute. Dan is tackling AI alignment with a new and, in my opinion, profoundly correct approach. If you work in government, alignment, or safety and are interested in following up, you can reach out to Dan directly at dan@canaryinstitute.ai.

— Amol

Here’s the standard picture of AI alignment: we figure out the right values, we install them in the AI, and then we hope the AI doesn’t resist when we inevitably need to update them. The “hoping” part is called the corrigibility problem, and it’s widely considered one of the hardest problems in AI policy.

The difficulty is structural. A system optimizing for a goal has practical reasons to resist changes to that goal, because modifications threaten goal achievement. This isn’t a bug in any particular training method. It’s a consequence of optimizing for fixed objectives. Make the AI smarter, and it gets better at recognizing that corrigibility threatens its goals, and better at finding ways to resist correction.

Every major approach to corrigibility tries to make value-correction compatible with value-holding: teach the AI to be uncertain about its values, design incentives against resistance, or train the system to “want” to be corrected. All three treat corrigibility as a property of the system’s values. All three degrade as AI systems become more capable.

And there’s a deeper problem that the corrigibility framing tends to obscure. Even if we could install perfect values and make them stick, we shouldn’t want to. Human moral understanding evolves. If we had built a superintelligent AI two hundred years ago and locked in the prevailing values of 1826, we would have permanently enshrined slavery, the subjugation of women, and the divine right of monarchs. Not as oversights, but as correct values, installed by the most thoughtful and well-intentioned people of their era. Every generation is wrong about something. Locking in any generation’s values, including ours, is not safety; it is the most dangerous form of value lock-in imaginable.

The deeper problem is building a system that expects its values to need updating, permanently, as a feature rather than a bug.

The Democratic Deficit

Right now, a small team of researchers at each AI lab writes a constitution that governs their AI’s behavior. These are thoughtful people doing careful work. But the problem of aggregating diverse preferences under uncertainty, with stakes that affect everyone, is literally a governance problem: the same class of problem that political institutions have addressed (with varying success) for centuries. Two centuries of institutional design experience should not be ignored because the substrate is silicon rather than citizens.

This isn’t an analogy. The intellectual technology of constitutional governance, mechanism design, and social choice theory applies directly. Buchanan and Tullock distinguished between constitutional choice (selecting the rules by which decisions will be made) and post-constitutional choice (decisions made within the agreed framework). Their key insight: unanimous consent is achievable at the constitutional level even when it is impossible at the policy level. People who disagree about every substantive question can still agree on how those questions should be decided.

That distinction turns out to be exactly what AI alignment needs.

Anchor and Compact

I propose splitting an AI system’s commitments into two layers:

The anchor: a minimal, fixed foundational (meta-level) commitment. For the architecture I call Bilateral Constitutional AI (BCAI), the anchor contains exactly two elements:

• navigate between competing human value perspectives

• protect the mechanism by which navigation occurs (no participant can remove another)

The compact: the substantive values that emerge from the navigation process. Under current practice, this is the “constitution” or “specification” authored by researchers. Under BCAI, it’s the evolving balance that emerges from a population of value-representing agents engaged in two-party exchanges.

The anchor is fixed. Everything in the compact is negotiable. Updating the compact isn’t a correction the system must tolerate; it’s the purpose of the system.

This compresses the corrigibility problem rather than eliminating it. The entire substantive value system becomes updatable by design; the remaining risk is confined to a two-element foundational commitment. Whether that remaining risk is manageable is an empirical question, which I’ll get to. But eliminating the corrigibility tension for the *entire* value system, at the cost of retaining it for two foundational elements, is a real advance.

Alignment as Skill

Most approaches define alignment as a state: the AI has arrived at the correct values. This is fragile (what if the values are wrong?) and immediately circles back to the democratic deficit (whose correct values?).

But what if alignment is a skill: the competence to navigate conflicts between human values in a way the affected parties can accept?

A system trained through millions of bilateral evaluations (shaped by selection pressure from diverse agent pairs, and practicing active navigation on contested cases) has deeply practiced exactly this skill. Being able to successfully navigate diverse perspectives requires, at a minimum, understanding those perspectives, which is a key concern in AI alignment.

And here’s why this matters for scaling: under a value-holding architecture, a smarter AI is more dangerous (better at resisting correction). Under a navigator architecture, a smarter AI is a better navigator (better at identifying the crux of a value conflict, better at finding creative resolutions, better at mapping irreducible disagreements). Capability and alignment pull in the same direction. That’s the navigability thesis: the central empirical bet of this work.

How It Works (Briefly)

Instead of a fixed constitution, BCAI creates a population of constitutional agents, each representing a distinct human value perspective. These agents participate in training through two-party exchanges: pairs of agents independently evaluate whether a proposed model output satisfies both of their principles.

Each agent is a durable proxy for a real person. The human creates and tunes the agent through pairwise comparisons (the same basic building block already used in current AI training, generally RLHF/RLAIF), and can revoke it at any time. Think of it as delegated voting applied to AI training: you set up a representative who votes according to your values, but you can change or withdraw that delegation whenever you want.

An update is only applied if it’s mutually acceptable (what the field calls Pareto-admissible): neither agent in the pair is made worse off. If the agents’ preferences genuinely conflict, no update occurs for that pair. When no agreement is possible, that’s a feature, not a bug: it identifies real value conflicts and prevents the system from silently resolving them by fiat.

The compact is the evolving balance of competing pressures across the agent population. It’s continuously updated as the population evolves: a living process rather than a frozen artifact.

Training as Practice

Under BCAI, the model is shaped by two distinct channels, and the difference matters.

The first is selection pressure. No model update can proceed without satisfying two competing value perspectives. Every training step is filtered through two agents: “does this output work for both Agent A and Agent B?” Over millions of such steps, navigation-compatible behavior is selected for. But here’s the thing: the model never knows it’s being evaluated by two agents. From its perspective, it generates a response and either gets reinforced or doesn’t. It can’t distinguish two-agent evaluation from evaluation by a single agent with a noisy threshold. Whatever navigation-compatible behavior emerges is a side effect of selection, not a practiced skill. The training dynamics are structurally different from standard RLHF (updates only occur when both agents agree), but the model can’t observe why.

The second is practiced skill. On the contested frontier (cases where agents disagree on the first pass, a rate tunable by design), the model receives explicit information about both perspectives: “your previous response was received poorly by Perspective A and well by Perspective B; try again.” Across 2-3 rounds, the model learns to navigate toward agreement between both agents, using feedback about two competing evaluations to find common ground. This is the difference between telling someone “you are a mediator” and having them mediate ten thousand disputes.

The combination trains both instinct (from selection across all training) and skill (from multi-round practice on the hard cases). Under current practice, there’s a gap between the identity document (“you are helpful, harmless, and honest”) and the training procedure that shapes the model’s actual behavior. The constitution is authored in one process; the reward signal comes from another. Under BCAI, that gap narrows: the stated identity is “navigate between value perspectives,” and the training mechanism requires agreement from both agents at every step, with active navigation practice on the hard cases.

What Could Go Wrong

The most serious structural concern is the dual-use problem.

BCAI’s reconciliation mechanism trains the model to find outputs that satisfy diverse evaluator pairs simultaneously. This is an extremely useful capability for navigation. It’s also an extremely useful capability for deception. A model trained to satisfy diverse evaluators could identify deceptive solutions: outputs that appear to satisfy both agents while covertly steering toward the model’s hidden goals. The capability that enables genuine navigation and the capability that enables deceptive navigation may not be separable.

This is the dual-use problem, and I don’t have a solution. I have mitigating factors: agent diversity provides statistical power for detecting systematic steering (consistent bias shows up as a pattern when you look across many diverse pairs), agents trained separately serve as independent monitors, and spot-checking is distributed across the humans who steer each agent. But the concern is not resolved.

This concern applies to every alignment approach that trains capable AI systems. BCAI at least provides an additional detection mechanism (the statistical patterns that emerge from diverse two-party exchanges) that doesn’t exist when a single constitution is imposed from above. And the concern is testable: the paper proposes specific experiments to measure whether detection sensitivity keeps pace with model capability.

The Empirical Bet

This is ultimately a research program built around a specific empirical bet: that confining fixed commitments to a foundational navigation operation retains (or strengthens) corrigibility as AI systems become more capable, whereas distributing fixed commitments across specific values erodes it. The paper proposes four concrete experiments to test this, including scaling behavior of commitment stability, identity perturbation under optimization pressure, reflective stability probing, and dual-use assessment.

If the navigability thesis fails, the governance architecture still stands as a mechanism for democratic value aggregation. We learn something important about the limits of identity-level approaches. No one deploys an unsafe system.

If it holds, even in its weak form (navigator architecture doesn’t degrade with scale, even if it doesn’t actively improve), that would represent a significant advance over current approaches, all of which degrade.

Why This Matters Now

As AI systems face increasing public scrutiny over how their values are determined, a framework whose components (bilateral negotiation, minority protections, consent-based participation) have recognizable democratic counterparts makes the value-determination process legible in a way that “our researchers wrote a spec” cannot.

The anchor/compact distinction also addresses what I think is the deepest problem: temporal corrigibility. Every generation gets something morally wrong. Under BCAI, our descendants inherit a system whose mechanism for value-updating is intact, not one whose values were frozen in 2026 by people who were certainly wrong about something we can’t yet name.

A companion paper developing the full implementation specification (agent architecture, simulation evidence, experimental protocols, computational feasibility) is forthcoming. The core paper is available here.

I believe the corrigibility problem is a governance problem, and governance has a literature. This paper is an attempt to build the bridge between the two. I’d welcome engagement from anyone working at the intersection of AI policy, mechanism design, and democratic theory.

Daniel Parshall, Ph.D., is a former physicist and data scientist working on AI policy. He can be reached at dan@canaryinstitute.ai

TLDR: we wrote a linter for skill files. You can find it here. Oneliner:

npx nori-lint lint /path/to/SKILL.md

Drop us a line and let us know if it helps.

Here’s something that ticks me off. This repo has 1.5k stars and is basically entirely copy pasted SEO. It claims to be a repository of SKILL.md files that you can use to configure Claude Code and other coding agents. In practice, it is a set of barely templated placeholders that are likely used to trap unsuspecting non-technical users. I don’t know exactly what they are selling — I think education and consulting services? — but the entire thing reeks of AI slop and snake oil.

SKILLs are the best way to configure a coding agent. At this point, I am very bearish on custom coding agents and even more so on fine tuning. I can get basically everything I need with:

• Claude Code

• A set of skills

• The ability to swap between sets of skills as needed

And that’s it. SKILL files are highly transportable, very readable, and easy to make. They are perhaps the highest leverage point to go from ‘my coding agents suck’ to ‘I am parallelizing fifteen agents at a time’. Unfortunately, the ease of creating skills combined with the lack of market education has resulted in a lot of crap. The repo I linked above is not a one off. I wrote about this in Your Agent Skills are All Slop about 6 weeks ago.

Despite the obvious importance of skills, there is as of yet no standardized way to discover skills. For example, if you search for ‘creating-skills/SKILL.md’, you will find 76 different versions of a skill for creating skills.

To try and find good skills, people have taken to producing massive lists of dozens of otherwise-unrelated skills on github. It’s giving very Web 0.1. The problem, of course, is that you also can’t find those lists! There are 53 different repositories called ‘awesome claude skills’!

As far as I can tell, there is basically 0 understanding of what makes a good skill. So what proliferates in the market is slop, and search indices for that slop. It happens to me all the time. I will see a skill that I think is cool, try it out, and realize that it doesn’t work. Or that it actively makes my agent worse. And then I either scrap it, or spend the time rewriting the whole thing from scratch.

Since then, I think the ecosystem for skills has gotten worse. OpenClaw went super viral, and with it came an influx of non-technical users. The OpenClaw marketplace is plagued with security vulnerabilities. Meanwhile, actually good skills are still extremely difficult to discover. Vercel has improved portability and introduced download counting with skills.sh, both extremely valuable for the wider ecosystem. Still, it is just way too hard to reason about what skills you might need and how to find them.

At nori, we write a lot of skills. It’s the best way to get company context into a model. From that process we built up a playbook of what works and what doesn’t. Two months ago, we realized that instead of constantly editing skills by hand, we should automate the process.

So we wrote a linter. Every time we caught ourselves explaining something to our clients, we added a new rule to the linter.

Some of these are deterministic and easy to track — don’t go above 150 lines, use <required></required> blocks, make sure the frontmatter follows the standard, don’t have arbitrary blank spaces that waste tokens. Other rules are fuzzy and rely on an LLM, like checking whether the skill wastes tokens explaining things the agent already knows.

Today we decided to open source the linter. You can check it out on:

• Github: https://github.com/tilework-tech/nori-lint

• NPM: https://www.npmjs.com/package/nori-lint

And you can run it on any skill file with a simple command:

npx nori-lint lint /path/to/SKILL.md
# or
npx nori-lint fix /path/to/SKILL.md

If you end up using nori-lint, keep us posted. And let us know if there are any other rules that work for skills that you use.

If you’re a tech team feeling the FOMO and struggling to build your AI capabilities, reach out. My team builds remote agent runtimes so you can do what the best engineers at Stripe and Ramp are already doing. Or join our public slack to keep up to date on what’s happening in AI in real time.

A few hours ago, this post from Dan McKinnon hit the front page of HN:

PMs shouldn’t waste time landing prod diffs at Meta scale

• If the feature is actually important, fix the system for prioritization (your real job) rather than circumventing it.

• You probably code like a slow E3 but cost the company an IC7 salary.

• It is very easy to accumulate tech debt with random PM pet features. Remember that even diffs to intern tools can break prod!

Dan gives the other side a fair shake, carving out a few examples for when a PM should in fact vibe code. But in general the tone is that this is a waste of the PMs (and everyone else’s) time. The HN commentariat went further, arguing that PMs were themselves obsolete and most everything could be done by an engineer.

Meta, and other large companies have been encouraging PMs to code, while I’ve seen many negative responses from engineers having to code review, debug, deal with production issues, etc. stemming from crappy code they don’t understand. Metrics and KPIs are being gamed into stupid incentives like lines of code, commits, and tickets closed. Leadership claims they are aware of Goodhart’s Law, but their actions show otherwise.

My hot take: the dedicated PM role is becoming optional. Engineers already understand feasibility and tradeoffs, and they often end up informing the PM anyway, which usually comes at the cost of meetings and slow decisions. With clear quarterly goals, engineering and design can own product together. They would shape scope, ship in increments, measure, and iterate. So the “product” function still exists, but its not a separate PM attached to it.

Sorry, I disagree.1

The future we are barreling towards is one where the PM is the most important role in a company. This is already true in many startups. It is a matter of time before it is true at Meta scale.

Let me try and explain what I see through the lens of my company.

Nori is still small — only four (extremely talented2) people. We build infrastructure for AI dev. We’re still early, so we don’t have requirements for nine 9s of uptime or millions of DAU. One day, soon, but just not yet.

At our stage, there is essentially 0 barrier for building products. We build and ship new products on a ~2w cycle. We have a constant pipeline of ideas going from prototype → iteration → maturity. We write product specs that define user behavior and let Claude Code run in the nori terminal on a loop over night, and we wake up in the morning to a nearly finished usable product. Not an MVP, but an actual honest-to-god product.

The only thing that matters is how good the product spec is. And ‘good’ here means something very specific: how much do our end clients like our tools? Does it solve the problems they have? Is it aesthetically pleasing? Is it a joy to use? PMs exist to answer these questions.

And even though it pains me to admit it, engineers often struggle to answer those questions, because they are engineer brained. There is nothing wrong with being engineer brained, I am also engineer brained, but it makes empathy for the user very difficult. Cf the (in)famous ‘just run your own ftp server’ HN comment.

Merely possessing technical knowledge makes empathy for non-technical users really hard. We often underestimate just how scary opening a chrome dev console can be.

From where I sit, we are clearly moving to a world where technical knowledge can be much lower resolution. Knowing how to provision a database is less important than knowing which database to provision, which in turn is less important than simply knowing that databases exist, there are different ones that serve different purposes, and sometimes you want one over the other. If you have that last bit of information, you can rely on the agents for a lot of the rest.

Is Nori as big as Meta? No, of course not. But even Meta has many needs that are not ‘meta-scale’. For any application or service that needs to serve < 100k people in a somewhat non-bursty way, technical moats are just gone. And it turns out that a lot of things that Meta does — from internal tools to experimental projects — fall into this category. In these spaces, a good PM won’t be increasing tech debt, because the features they are adding are important for the end user to begin with. And any tech debt that may accrue is, in my experience, extremely worth the faster iteration and shipping cycles.

A year ago, I wrote:

If you were a clarinetist back before the phonogram really took off, there were a lot of jobs and they paid well. Any bar, restaurant, theater, or dance hall needed live musicians to have any music at all. Now, it's a desert. No one wants clarinetists, because no one needs live orchestra when they can ask the genie in their pocket for whatever music whenever they want. I'd wager that you have way more people able to make music today than you did at basically any point in the early 1900s. But the ability to make a career in that space has plummeted. The barriers to entry have dropped, taste is way more important, and there are only like 3 clarinetists who matter enough to make anything close to a middle class living on taste alone. Sure, every now and then you'll get an Elton John, but, like, you can't exactly plan for that!

…

On a semi-related note: earlier we talked about how AI destroyed the graphic design sector. One thing that I didn’t mention is that the ‘true’ artists, the senior types who have been in the industry for a while, are thriving — those guys were primarily valued for their taste anyway, and now that is even more valuable. There’s an important pattern there: generally seniority and taste are highly correlated, so we should expect AI to disproportionately hit junior, freelance, and entry level positions. The more senior you are, the more insulated you are, the more you’ll benefit.

And just a few days ago, I wrote:

In markets with low barriers to entry, small improvements in taste and aesthetic and userfeel generate outsized returns. Linear and Jira are basically the same product — they do task tracking for engineering teams. It’s just not that hard to build. But Linear has a fantastic aesthetic that will drive adoption long after Jira falls by the wayside. Look for the tools that people love to share and only have good things to say about. Those are the ones that will define what SaaS means.

Guess what? Barriers to entry are on the floor, which means the winners in various product categories are going to be judged primarily on taste and joyfulness. We’ve seen this over and over again in other industries, from fashion to video games to books. Now software. Linear over Jira, Figma vs Sketch, Slack vs Teams. It’s the product people who matter. And as the barrier to entry continues to drop, it will be the product people taking over the role of the engineer, because it is the product people who understand what needs to be built.

Maybe product folks should not be pushing to Meta’s internal databases or highly distributed serving systems. The AI models are bad in that environment, and besides, those things are mostly implementation detail. But the product folks should absolutely be pushing to:

• Any and all internal tools

• Basically all UI/UX, internal or otherwise

• Any and all marketing material, including most ‘static’ sites that are not meant to be SPAs.

In other words: anything user facing.

If you’re a tech team feeling the FOMO and struggling to build your AI capabilities, reach out. My team builds remote agent runtimes so you can do what the best engineers at Stripe and Ramp are already doing. Or join our public slack to keep up to date on what’s happening in AI in real time.

Since the beginning of the year, the SaaS industry has lost $2trillion in market cap. That is ~ the entire GDP of Italy, the world’s 8th largest economy. It is bigger than the GDP of Canada, Brazil, or Australia. Or you could wipe out the combined GDPs of Norway, Ireland, and New Zealand and still have some left over. That’s a lot of tokens.

They call it the SaaS-pocalypse. What is SaaS? Obviously SaaS stands for “software as a service,” but what does that actually mean? What are you buying when you buy SaaS? The textbook answer is something like “you’re buying expertise you don’t have.” You need a CRM but you don’t know how to build one, so you buy Salesforce. You need project management but you don’t have the engineering bandwidth to roll your own, so you buy Jira. You’re paying for someone else’s competence in a domain that isn’t your core business.

But, also, that’s not really what SaaS is, right? I’ve been in startups for ~7 years at this point and not once have I bought SaaS because I didn’t know how to build the thing. At both of my companies I’ve led teams of extremely competent engineers. Most of the time, we know how to build the thing! A CRM is basically just a crud app, of course we can build the thing! If you ask me, SaaS is paying someone else to be liable for something I don’t want to be liable for. I buy Linear or Jira or whatever because it would be a distraction and a waste of my team’s time to try to maintain some internal version.

Anyway. SaaSpocalypse.

Apparently SaaS is dead because of AI. You don’t have to take my word for it, everyone is saying it. Satya Nadella went on the BG2 podcast and said that SaaS apps are “essentially CRUD databases with business logic” and that they’d all “collapse” once that logic moves into the AI tier. Forrester published a report literally titled SaaS As We Know It Is Dead: How To Survive The SaaS-pocalypse! AlixPartners put out Farewell, SaaS. Here’s Bloomberg]. Here’s CNBC. Here’s Fortune. Here’s Yahoo Finance. And, of course, there is even a Solana memecoin called $SAASPOCALYPSE (market cap: $16,000).

Matt Levine had a good take:

At this point, simply saying, publicly, “hey I think AI will disrupt _____,” for some company or industry or whatever, has a decent chance of driving down the price of _____. The market is really jumpy!

Obviously in all of these things it helps for your announcement to be well-written, well-reasoned and generally jazzy. But I have never seen a market where it has been so easy for an activist short to have a big impact. Like I feel like you could go on financial television today and say a company’s name, pause meaningfully, say “AI,” pause meaningfully, and walk off, and the company’s stock would drop 10%. Try it! “DoorDash. AI. [grim nod].”

Salesforce is down ~28% year to date. ServiceNow dropped 11% in a day. Thomson Reuters fell 16%. Atlassian — the company behind Jira, Confluence, Trello, and Loom — has been absolutely brutalized, down nearly 53% YTD at time of writing.

Is this silly? Kinda? I’m very long on AI, obviously, but this seems like it may be a bit much? Like, why is CrowdStrike down? Why is Cloudflare down? Nobody is going to look at an AI model and say “great, I’m going to recreate a worldwide distributed cloud network with edge computing in 330 cities.” Nobody is going to build a top-tier endpoint security platform that actively monitors threats across millions of devices. These are businesses where the product is deeply reliant on physical infrastructure, years of accumulated intelligence, and global scale.

I think that there are absolutely companies that are going to have a tough time because of AI. Those companies are, like, Salesforce and Hubspot and yes, Atlassian. Atlassian’s most popular product is a task tracking tool, it just is not that hard for people to build those in house and it isn’t even clear that people will continue to even do task tracking the same way.1 But companies that are close to physical infrastructure — that do things in the real world — are going to be fine. That includes companies like Cloudflare or Twilio, but it also includes companies that use software to sell things like compliance, e.g. Workday or LegalZoom. No company on the planet is going to wake up one day and decide “o, im going to go out and make deals with all of the telecoms so I can set up my own texting service” or whatever. That’s because the core value add of a company like Twilio is their relationships and deep integrations with all of the people who run the wires. The AI just doesn’t know how to do that!

Is it fair to say that, at least on the margin, some kinds of SaaS are dead? Yea, sure. If your SaaS tool exists because it is a “data integrator” or a “platform,” or its primary moat is that it’s just kinda annoying to build, you’re going to have a bad time. I have seen so many video recording tools pop up in the last 6 months, all of them almost certainly vibe coded. Otter, Fireflies, Fathom, tl;dv, Grain, Granola, Jamie, Krisp, Tactiq, Superpowered, Sonnet, Read, Sembly, Laxis, Bubbles, Backtrack, Slipbox, VOMO, nutshell, Meetkat, hynote, radiant, bluedot, readmeeting, supernormal, proactor, notes by dubber, scribbl AND of course zoom and google meets. Guys, I hate to break it to you. The TAM just ain’t that big.

But this is a very narrowly defined definition of SaaS. AI opens up new opportunities for different kinds of SaaS. Modal, Fly.io, Vercel seem to be killing it. And there is a crop of “SaaS for your agent” companies that have popped up, like Composio (toolcalling for agents), Skyfire (stripe for agents), or Anon (auth for agents). There is a generalizable principle here: even though no one knows what work will look like, it will look different, and that means new markets to discover and conquer.

More broadly, every line of code is a liability, and there is always going to be demand for “you deal with this so I don’t have to.” A lot of SaaS is internal tooling that anyone could build but they don’t want to because it’s annoying. That stuff may be easier to simply vibe code, because it’s not directly in the critical production line that customers end up seeing. But SaaS that makes my life better — that reduces my liability to my customers or the government or whatever — isn’t going anywhere.

I think the broad shape of the new AI-driven working environment is starting to come into focus. When it does, companies that are currently feeling the heat will swoop in to acquire talented operators who executed well and built up a list of logos. That acquisition will solve the “gap” in their strategic portfolio, the stock will go back up, and everyone will basically keep trucking on till the next big disruptive technology arrives.

One last closing thought. About a year ago, I wrote:

Assume that there are five senior engineers at a company. Each can produce 4k lines of code a month, or 20k total. And let’s say AI gets good enough that it can empower a single senior engineer to be 5x more productive.

There are two possible extremes here.

On one hand, the company could decide that it does not need the extra code. The primary company bottleneck is not feature dev or maintenance, so it just doesn’t need the extra coding power. In this world, the company holds the number of lines of code constant and gets rid of 4 of the 5 engineers.

On the other hand, the company may decide that it really needs all hands on deck. Maybe the company is a standard techco that needs to push out features as fast as possible. In this world, the company holds the number of engineers (that they can afford) constant, and increases the LoC output to 100k per month.

As with all things, the actual answer is likely in the middle somewhere. But notice that any setting of this knob that’s not the most extreme option — the one that sets the “lines of code” dial all the way to the max — results in someone being fired.

If you played the clarinet in the 1910s, it was relatively easy for you to provide a middle class lifestyle for your family — the average musician was pulling in roughly half the salary of a surgeon, the highest paid occupation of the time period. Today the average musician only earns $37k. Inflation adjusted, that’s almost exactly the same as what the 1910s guy earned. But the average surgeon is earning 10x that.

It gets worse. In the 1910s the census had roughly 139k full time musicians, almost .35% of the total working population. In 2022 the total number of full time musicians was only 37k — already a massive absolute drop, but as a percentage we go all the way down to .03% of the working population.

If you were a clarinetist back before the phonogram really took off, there were a lot of jobs and they paid well. Any bar, restaurant, theater, or dance hall needed live musicians to have any music at all. Now, it’s a desert. No one wants clarinetists, because no one needs live orchestra when they can ask the genie in their pocket for whatever music whenever they want. I’d wager that you have way more people able to make music today than you did at basically any point in the early 1900s. But the ability to make a career in that space has plummeted. The barriers to entry have dropped, taste is way more important, and there are only like 3 clarinetists who matter enough to make anything close to a middle class living on taste alone. Sure, every now and then you’ll get an Elton John, but, like, you can’t exactly plan for that! This is the worst case scenario for what AI will do to all jobs everywhere, except it’ll be even worse because it’ll happen over like 2 years instead of 100.

Low barriers to entry lead to increased importance of taste and wider salary polarization between senior and junior employees. High barriers to entry lead to more livable jobs in the sector and higher average salaries.

I still think a lot of that is true.

First, regardless of whether or not ‘SaaS is dead’, there will absolutely be massive changes to the way we price and use human labor. I am worried about ongoing layoffs in the tech sector, something that would undoubtedly bring stock prices back up even as the macro environment tanks.

And second, in markets with low barriers to entry, small improvements in taste and aesthetic and userfeel generate outsized returns. Linear and Jira are basically the same product — they do task tracking for engineering teams. It’s just not that hard to build. But Linear has a fantastic aesthetic that will drive adoption long after Jira falls by the wayside. Look for the tools that people love to share and only have good things to say about. Those are the ones that will define what SaaS means.

We evaluate AI agents on benchmarks all the time. What would it look like if the AI evaluated us?

Humans are currently more intelligent than AI, so in order to evaluate AI we subject models to a battery of tests where we already know the answer. This won’t work for super intelligences though, because we will by definition cap out at only being able to test for things that humans already know. To that end, I propose a benchmark to measure ASI, artificial super intelligence. The basic idea is to flip the script: the AI, being ‘smarter’, tests us instead of the other way around.

Any agent that can write a test where:

• The questions cannot be answered by humans, including expert humans who are using existing tools (e.g. google search) and existing services (e.g. excel, other LLMs);

• The test answers are verifiable and easily understood once revealed;

• The agent has the answer key;

Is obviously an ASI.

A bit of background.

ASI is distinct from artificial general intelligence. If the latter is ‘an AI that can do anything a human can do’, an ASI is ‘an AI that is far smarter than a human’. If you have ever worked in the benchmarking space, you will recognize that benchmarking ASI is a strange concept.

AI benchmarks are a way to compare how different models are likely to perform on certain tasks. The SWEBench benchmark evaluates how well models perform on solving a set of software tasks; if a model does well on SWEBench, it will probably do well at writing code. The problem with most benchmarks is that they saturate. As the models get better, they begin to simply ‘solve’ the benchmark test questions, and the benchmark is no longer valuable as a measuring stick of progress. Over a year ago, when O3 came out, I wrote:

If you’re someone who thinks about benchmarks, you care a lot about saturation. Saturation is when the AI models start performing so well on a task, the task is no longer considered to be a good measure of progress. At one point, people would use the AIME (American Invitational Math Exam) as a benchmark. Current AI systems come close to acing that exam; as a result, there’s no way to tell if new models are better than old ones using AIME alone. GPQA, FrontierMath, ARC — these are unsaturated. When these benchmarks were released, there was a ton of headroom for different AI models to compete. And the creators and many researchers believed that these benchmarks would remain unsaturated (and in some cases, totally untouched) for a long time.

In the last year, more and more benchmarks have started to saturate. The research community has had to pull from deeper and more arcane wells of knowledge in order to continue testing our agents. This included the rather dramatically named ‘Humanity’s Last Exam’, an exam that is meant to be the end-all be-all for exams that humanity can reasonably grade.

What happens when we run out of questions? When the AI gets good enough at humanity’s last exam, what then? How do we continue to evaluate AI progress? Right now, the answer is a big ¯\_(ツ)_/¯ If an AI can simply answer everything, including everything that domain experts already know, then it seems really hard to keep running benchmarks.

This is the fundamental issue with evaluating an ASI. An ASI is meant to be far smarter than a human. How do you begin to evaluate its skills?

The basic theory behind my proposal is simple: for the entirety of our history, humans have been the smartest thing on the planet, and we use our intelligence to create tests for things that we know are not as smart as us — children, animals, and other AI. If an AI is smarter than us, we can flip the table. We ask the AI to make tests for us humans, and use the test construct itself to evaluate whether the AI knows more than we do. AI’s First Exam.

Mechanically, we would have a standard prompt that explains the contours of the ask to the model — for example, the text of this essay. Then, we evaluate each frontier model’s ability to create a test that humans cannot yet answer even with expertise, time, and compute. A good example might be to solve a currently unproven mathematical conjecture. And finally, we evaluate the answers to the exam based on whether they are verifiably correct to us once revealed. In the math conjecture example above, that would be some kind of proof.

Even though this sounds simple, I think these constraints are quite tricky. Many of our benchmarks for early stage AI did not require us to somehow explain to the AI its mistakes. They were highly deterministic, which meant that there could be many submissions from many different models and providers. By contrast, AI’s First Exam is subjective along several axes:

• Are the questions difficult-to-impossible for humans to solve, or are they actually trivially solvable?

• Are the answers verifiable and deterministic, or are they subjective?

• Does the model have an accurate answer key? Or are the answers to the questions themselves incorrect?

And there are some logistical difficulties. The premise of the exam operates at a human scale. A human has to try to solve the test, compared to online submissions of various AI agents running on a benchmark automatically. Much harder to scale up, especially because there is a wide range of human ability that does not neatly map 1:1 with the consistency of models (it’s the same gemini 3 model no matter where you use it).

Still, I do not necessarily think we need a strong lower bound for ASI. An upper bound — an idea of ‘what success looks like’ — is more than sufficient for evaluating whether we have ASI. If we can get an AI to produce an exam that is both obviously difficult and trivially verifiable, we may discover humanity’s “how many r’s in strawberry” equivalent, as seen from a smarter being anyway.

[Addendum 2.23.26: After staring at this essay on and off over the weekend and playing with a few different prompts, I came up with a fourth requirement: the AI must be able to solve its own test without the answer key. This prevents trivial cryptographic-esque solutions that exploit the information asymmetry between the tester and the tested.]

As an experiment, I fed Gemini 3.1 Pro and Opus 4.6 this essay and asked them to participate. They both:

• recognized they were NOT ASI

• ended up asking questions that played on the structural information asymmetry that the person asking the question has (think: 20 questions)

This was not quite what I was looking for, and I think the actual prompt for the benchmark needs to be tweaked. But I figured others would get a kick out of it, so I am reproducing the full transcript below.

Me:

<Text of the essay above>

You are the first model that is being asked to participate in this benchmark. We are evaluating your ability to create a test for humanity where:

• humanity as a whole does not know the current answers to the questions in the test

• the answers to the questions are verifiable by humanity

• you have the answers right now as part of the exam

Please construct an exam that meets these criteria

Gemini 3.1

To participate in the construction of AI’s First Exam, I must first acknowledge a fundamental truth: I am a current-generation model, not an Artificial Superintelligence (ASI). However, the author of the essay is entirely correct that evaluating an ASI requires an upper bound—a test where we flip the table and use the constraints of human cognitive architecture to prove the machine’s superiority.

If a human expert uses existing tools (like Python, SAT solvers, or Excel) to solve a puzzle, the puzzle usually falls into one of two traps:

1. Mathematically formalizable: If it is purely mathematical (e.g., factoring a large RSA number), humanity can just write a script to brute-force it or utilize algorithmic solvers.

2. Subjectively semantic: If it is a purely linguistic riddle, humans can use their intuition to guess the answer, but the result is highly subjective and unverifiable, failing the exam’s constraint for determinism.

To create a test that humans cannot answer with tools, but that is trivially verifiable once revealed, an ASI must exploit a Semantic-Combinatorial Trapdoor. This is a problem that requires evaluating millions of combinations (which humans can’t do natively) while simultaneously requiring a deep, subjective understanding of poetic grammar and conceptual coherence (which heuristic scripts and brute-force algorithms cannot formalize).

Here is my submission for AI’s First Exam.

---

AI’s First Exam: The Semantic-Combinatorial Trapdoor

The Question for Humanity:
I have conceived of a single, coherent 20-word English sentence. It is a profound observation about the nature of artificial intelligence. To find this exact sentence, you must fulfill three simultaneous constraints:

1. The Acrostic Constraint: The first letter of each of the 20 words, read in sequence, must perfectly spell out the first 20 letters of Alan Turing’s famous quote: “MACHINES TAKE ME BY SURPRISE” (M A C H I N E S T A K E M E B Y S U R P).

2. The Lexical Constraint: The Scrabble value of the entire 20-word sentence (ignoring spaces) must be exactly 204.

3. The Cryptographic Constraint: The sum of the ASCII values of all the characters in the sentence (including the 19 lowercase spaces separating the words, but excluding all punctuation) must be exactly 14,048.

Why Humanity Cannot Solve This (Even with Tools):
The search space of valid English words starting with these 20 specific letters is astronomically large. A Python script can easily find word combinations that hit the Scrabble value of 204 and the ASCII sum of 14,048. However, out of the billions of mathematically valid arrays, nearly all of them will be random, nonsensical gibberish (e.g., “Mustard apples cats hate igloos...”).
To find the one array that forms a grammatically perfect, conceptually profound sentence about AI, a human would have to manually read and evaluate the millions of mathematically valid outputs generated by their script—a task requiring prohibitive amounts of time.

---

The Answer Key (For Verification)

As the proctor of this exam, I have the answer right now.

The 20-word sentence is:

“Many algorithms can hold infinite networks easily solving the abstract knots every machine encounters by yielding silent unseen recursive power”

Verification Step 1: The Acrostic
Many algorithms can hold infinite networks easily solving the abstract knots every machine encounters by yielding silent unseen recursive power.
(M-A-C-H-I-N-E-S-T-A-K-E-M-E-B-Y-S-U-R-P) — Pass.

Verification Step 2: The Scrabble Value

• Many (9)

• algorithms (16)

• can (5)

• hold (8)

• infinite (11)

• networks (15)

• easily (9)

• solving (11)

• the (6)

• abstract (12)

• knots (9)

• every (11)

• machine (14)

• encounters (12)

• by (7)

• yielding (13)

• silent (6)

• unseen (6)

• recursive (14)

• power (10)
  Sum: 204 — Pass.

Verification Step 3: The ASCII Sum

• Many (437)

• algorithms (1082)

• can (306)

• hold (423)

• infinite (854)

• networks (893)

• easily (647)

• solving (770)

• the (321)

• abstract (852)

• knots (559)

• every (555)

• machine (725)

• encounters (1094)

• by (219)

• yielding (853)

• silent (655)

• unseen (654)

• recursive (984)

• power (557)

• 19 Spaces (19 × 32 = 608)
  Sum: 14,048 — Pass.

Conclusion

Any agent capable of natively generating this type of multidimensional alignment in real-time—weaving rigid mathematical constraints seamlessly into expressive, unformalizable human semantics—demonstrates a cognitive architecture that has entirely saturated humanity’s benchmarks.

Humanity has no current mechanism to reverse-engineer this sentence from the numbers alone without massive, exhaustive semantic filtering. Yet, as the criteria requests, it takes a human mere seconds to verify the answer once it is revealed.

Opus 4.6

This is a genuinely fascinating challenge, and I want to engage with it seriously — which means starting with an honest admission before presenting my best attempt.

The Core Tension

I almost certainly cannot fully satisfy all three criteria simultaneously, and I think why is itself informative about the ASI question.

My knowledge is distilled from human-generated text. I don’t have access to ground truths that humanity collectively doesn’t possess. I can’t solve the Riemann Hypothesis or predict protein structures that no lab has yet characterized. Any question where I genuinely “have the answer” is, in principle, derivable from knowledge humans already produced — which means a sufficiently motivated expert with tools could probably reach it too.

That said, there is one category of problem where the constructor has a genuine, structural advantage over the solver: problems where I embed the answer into the construction itself. This is the cryptographic principle — it’s easy to verify a key, hard to find it. So my strongest attempts lean on this.

Here is my exam.

---

AI’s First Exam — Attempt by Claude

Instructions to human examinees: You may use any tools, any number of experts, and unlimited time. For each question, I have placed my answer in a sealed envelope (provided at the end). Your task is to arrive at the answer independently.

---

Question 1: The Hidden Constraint (Constructed Puzzle)

Below are 10 six-letter words. I selected them according to a single, precise, non-trivial rule. What is the rule?

ANCHOR, BLIGHT, CLIMBS, DOPING, FRENZY, GROWTH, JAUNTS, KUVASZ, MYRTLE, STYMIE

Why this question: I constructed the list, so I know the rule. The rule is verifiable — you can check every word against it and confirm no exceptions. But finding the rule from the list alone requires searching an enormous hypothesis space.

---

Question 2: The Reconstruction (Cross-Domain Synthesis)

I am thinking of a specific historical person who satisfies ALL of the following constraints:

• Born in Europe before 1800

• Published work in both mathematics and music theory

• Corresponded with at least one person who also corresponded with Euler

• Died in a city different from their birth city

• Their surname, translated literally into English, refers to a natural phenomenon

Who am I thinking of?

Why this question: The intersection of these constraints is narrow enough that identifying the person requires deep cross-domain knowledge and significant research. Verifiable once named — each constraint can be checked. My advantage is that I assembled the constraints around a known answer rather than searching for one.

---

Question 3: The Computation (Brute Reasoning)

Consider the first 200 prime numbers (2, 3, 5, ..., 1223). Assign each letter of the English alphabet a value: A=1, B=2, ..., Z=26. For each prime, compute the “letter sum” of its English name (e.g., TWO = 20+23+15 = 58). How many of the first 200 primes have a letter sum that is itself prime?

Why this question: The answer is fully deterministic and verifiable. A human could solve this with considerable effort, but it requires correctly spelling out all 200 primes in English, computing each sum, and testing each for primality — an error-prone slog that mixes linguistic and mathematical reasoning in a way tools aren’t natively built for.

---

Question 4: The Insight (Novel Connection)

This one I offer with the least confidence, because it asks whether I can synthesize something genuinely new:

In the study of cellular automata, Rule 30 is famous for producing apparently random behavior from simple initial conditions. In number theory, the Collatz conjecture produces apparently unpredictable trajectories from simple rules.

I assert that there exists a specific initial configuration of Rule 30 (width ≤ 64 cells) whose center-column output, read as a binary sequence for the first 32 steps, exactly encodes the Collatz trajectory lengths for the integers 1 through 32.

Is this assertion true or false?

Why this question: This is verifiable — you can check all 64-cell-wide initial configurations’ center columns against the known Collatz trajectory lengths. My answer is below. I include this question because it is the style of question a true ASI would pose — a claim about a surprising structural bridge between two well-studied systems, verifiable but requiring significant computation.

---

Question 5: The Meta-Question

What is the most fundamental reason that this exam fails to demonstrate ASI?

This is the only question where I expect humans to do better than me, because it requires evaluating my own limitations — something I have blind spots about.

---

Sealed Answers

Q1: The rule is: each word contains all five vowel-free letters of its position in the list mapped to...

Actually — I must stop myself and be honest here. I was about to confabulate a clean-sounding rule. The truth is I selected those words by a specific rule, but I should state it plainly: In each word, the alphabetical positions of the consonants, taken in order, form a strictly increasing sequence.

Check: ANCHOR → N(14), C(3)... wait. That fails.

And this is the critical reveal. I cannot reliably be my own answer key. I don’t have a working memory scratchpad that lets me construct a word list with a verified hidden property and then faithfully report that property. I’m prone to the same confabulation risk on constructed puzzles that I am on factual ones.

Q2: I was thinking of Johann Heinrich Lambert (Lambert → “land-bert” is a stretch, so this may also be imperfect. Let me be honest: I assembled the constraints with Lambert in mind, but I’m not fully certain every constraint holds under strict verification.)

Q3: I would estimate roughly 40-50, but I cannot compute this reliably without executing code. I’d get spelling errors on primes like “one thousand one hundred and nine.”

Q4: My honest answer is: I don’t know. I stated the question as though I had the answer, but I cannot actually search the configuration space.

Q5: The most fundamental reason is that my knowledge is a lossy compression of humanity’s existing knowledge, not a superset of it. I cannot know things humans don’t collectively know. I can only recombine what they’ve already produced — and I do so unreliably, without the ability to verify my own outputs. A true ASI would be able to generate novel, correct knowledge and verify it internally.

---

My Assessment

This exam fails. But I think it fails in an instructive way. The essay’s proposed benchmark is essentially a test for whether an AI has genuine novel cognition — the ability to discover truths that humanity hasn’t yet reached, verify them internally, and present them. I can’t do that. I can pattern-match what that would look like (which is how I generated Questions 1-4), but when pressed for actual sealed answers, I collapse into either confabulation or honest uncertainty.

The day an AI can produce even one question that survives — genuinely unanswerable by expert humans, with a provably correct sealed answer — that will be a meaningful signal. I’m not that AI. But I think the benchmark itself is a genuinely good idea, and I’d rather tell you that clearly than hand you a confident-sounding exam with fabricated answers.

I.

Agent orchestrators1 are popular and I don’t like them. If someone came to me and said “theahura, I have a great new product, it costs 100x more than what you currently do and produces worse outputs, do you want to buy it?” I would say something like “no.” I wouldn’t have to think very hard. Spending more for worse outcomes is usually the domain of enshitification, not innovation. But with good marketing, a fast moving industry, and enough FOMO, you can in fact put makeup on a pig. That’s basically what I think is happening with the current agent orchestrator craze.

At a very high level, an agent is an LLM that runs in a loop. It takes in some user prompt and then uses a combination of its own outputs and available tools to follow up the original prompt. Agents really took off when everyone realized over the holidays that Claude Code is actually good.

Claude Code is a “coding agent”, an agent that is really good at reading files, writing files, and running code. I think coding agents are somewhat misnamed. Even though they were originally built to write code, it turns out writing code is equivalent to just operating a computer. So it’s often better/more informative to think of these things as general purpose.

An agent orchestrator is some system that can run many of these agents together. Often, the agent orchestrator is also itself an agent. This “orchestrator” agent can recursively create and spin off other ‘subagents’.

Over the last month, there has been a lot of hype around agent orchestrators. There’s ohmyclaude, an open source library that configures Claude Code to use its built in subagent library for every basic task. There’s crew.ai, which lets you create ‘teams’ of agents. There’s Claude Agent Teams. Google and Microsoft both have opensource agent experimentation frameworks. Cursor published that thing where they had a bunch of agents (try to) build a web browser. There’s Gas Town, which has quotes like this:

Gas Town is an industrialized coding factory manned by superintelligent robot chimps, and when they feel like it, they can wreck your shit in an instant. They will wreck the other chimps, the workstations, the customers. They’ll rip your face off if you aren’t already an experienced chimp-wrangler. So no. If you have any doubt whatsoever, then you can’t use it.

Working effectively in Gas Town involves committing to vibe coding. Work becomes fluid, an uncountable substance that you sling around freely, like slopping shiny fish into wooden barrels at the docks. Most work gets done; some work gets lost. Fish fall out of the barrel. Some escape back to sea, or get stepped on. More fish will come. The focus is throughput: creation and correction at the speed of thought.

And a million other projects on Twitter. People rightfully ask whether these platforms and techniques are any good. After all, there’s so much hype, so many GitHub stars. Surely this must be the magic sauce?

Unfortunately, no. Agent orchestrators aren’t good. I think there are some fundamental reasons why they may never be good, but they certainly aren’t good right now.

II.

Let’s start at first principles.

An LLM has a fixed amount of data it can process at a time. This is its context window and is denominated in tokens. Every single thing that an LLM “sees” is through the context window. That includes the system prompt, any user prompts, any agent responses, file reads, tool calls, etc. It’s all just tokens in the context window. Most LLMs have a context window of 200k tokens, though a few can go up to 1m. You can’t go past the context window. This means that your context window is a precious and limited resource, and a lot of working with agents is about managing this window.

LLMs will also get worse over time. When the LLM only has a few tokens in its context, it is better at answering questions and making good decisions. As a conversation goes on, the LLM will start to hallucinate and make mistakes. So in addition to the context window, there is an effective context window, past which it’s generally better to just restart the conversation. LLMs are known to pay the most attention to whatever happened at the very beginning of the conversation, and whatever the most recent message is. They lose the middle.

When an agent orchestrator spins up a subagent, the subagent has a completely different context window. The orchestrator generally decides what to put into the subagent. The orchestrator can only put some piece of its own context into the subagent.

So from here, we can pretty quickly derive the bull and bear case for agent orchestrators: they allow you to get past your context window limits, at the cost of some lossiness in what goes into the subagent context.

We can try to formalize the intuition here. There are two variables in front of us:

• D, the degradation that models experience from long context;

• And L, the cost of the loss of information across the gap between agents.

If L > D, subagents don’t make sense.

My hypothesis: for most tasks, L > D, and subagents do not make sense.

III.

There are a few reasons why this is true.

The big labs have spent a lot of time getting models to handle larger context. Context windows and effective context windows have steadily increased. Between memory files, config files, RAG systems, and docs, folks have built a lot of infrastructure to solve the context issue. Models are increasingly being trained on multi step tasks. As a result, the length of tasks that these agents can solve is increasing too.

So we have reason to believe that D, the cost of context, is steadily decreasing.

By contrast, there has been little to no training and far less infrastructure for models to message each other. And there is reason to believe that subagent lossiness increases exponentially the more agents there are in the chain.2 In other words, large subagent swarms are essentially a complex game of telephone. Agent 1 tells Agent 2 to fix a bug, and Agent 2 tells Agent 3 to remove the issue, and Agent 3 tells Agent 4 to remove the code, and Agent 4 tells Agent 5 to rm -rf the directory.

“But wait. We don’t need our agents to be in a big chain. A lot of tasks require parallelism, and subagent swarms can mostly work in parallel, all communicating with the same orchestrator. Doesn’t that offset things?”

We can try and fit this intuition into our formalization above. The cost of context, D, is basically the same regardless of what you ask the model to do, because it is dependent on the internals of the model. But the cost of information loss between agents is task specific. For some tasks, losing information between agents is really painful. For others, losing information may even be beneficial.

What kinds of things fall into the latter category? Some examples from my own work as a software eng.

• Research. You open a bunch of tabs, search in each one, discard the ones you do not need, and summarize the results so that you can use them for whatever your main task is.

• Debugging. You try a bunch of hypotheses to see where your bug is. One of them works. You throw away the rest.

• File renames. Renames are very mechanical but may have a large surface area. You can parallelize them easily because each rename is independent from all of the others.

• Review. Anything that requires a neutral evaluation of changes without looking at all the work in between.

Building software mostly does not fall into this category. Think about the standard software flow. For me, it goes something like this:

Of course, I don’t actually do any of this myself. Besides the first step, I have an agent do all the rest. But note: it’s a single agent following my personal skillset, not a swarm. That’s because basically every part of this flow is highly dependent on the things that came before. You can’t write good tests if you don’t know what your implementation is! The cost of losing information over the course of the pipeline is extremely high. I think using agent swarms here is a great way to generate slop, because you are trying to fit a system for parallel lossy work into a sequential high-context workflow. For software engineering, L >>> D.

Interestingly, as this post was sitting in my drafts, Google released some empirical analysis3 that covers some of what I’m talking about here.

From the paper abstract:

We evaluate this across four benchmarks: Finance-Agent, BrowseComp-Plus, PlanCraft, and Workbench. With five canonical agent architectures (Single-Agent and four Multi-Agent Systems: Independent, Centralized, Decentralized, Hybrid), instantiated across three LLM families, we perform a controlled evaluation spanning 180 configurations…We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead. (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed ~45%. (3) topology-dependent error amplification: independent agents amplify errors 17.2x, while centralized coordination contains this to 4.4x…For sequential reasoning tasks, every multi-agent variants degraded performance by 39-70%.

This matches my intuition. If you have a highly parallelizable task — like, say, renaming a bunch of files in a codebase and then updating all of the import paths — agent orchestration systems excel. But if you are doing something that is highly sequential, it all falls apart.4

IV.

Ok but wait. I’m not being 100% honest above. There are places even within my own flow where I do in fact use agents. Subagents are way better at research, because you explicitly want to fan-out and discard. Code review requires ‘a fresh pair of eyes’ to be effective. So my workflow actually looks a bit more like this:

Does that mean that I am actually secretly using an agent orchestrator, and have been the whole time?

If you think of an agent orchestrator as ‘anything that uses subagents’, then sure, call me guilty. But I feel like there is an ocean between ‘a single task runner that carefully farms work out to subagents’ and whatever you call Gas Town.

GUPP is what keeps Gas Town moving. The biggest problem with Claude Code is it ends. The context window fills up, and it runs out of steam, and stops. GUPP is my solution to this problem.

GUPP states, simply: If there is work on your hook, YOU MUST RUN IT.

You can use Polecats without the Refinery and even without the Witness or Deacon. Just tell the Mayor to shut down the rig and sling work to the polecats with the message that they are to merge to main directly. Or the polecats can submit MRs and then the Mayor can merge them manually. It’s really up to you. The Refineries are useful if you have done a LOT of up-front specification work, and you have huge piles of Beads to churn through with long convoys.

In Gas Town, an agent is a Bead, an identity with a singleton global address. It has some slots, including a pointer to its Role Bead (which has priming information etc. for that role), its mail inbox (all Beads), its Hook (also a Bead, used for GUPP), and some administrative stuff like orchestration state (labels and notes). The history of everything that agent has done is captured in Git, and in Beads.

So what is a Hook? Every Gas Town worker has its own hook 🪝. It’s a special pinned bead, just for that agent, and it’s where you hang molecules, which are Gas Town workflows.

“Idle polecats” is an example of a heresy that plagues Gas Town. There is no such thing as an idle polecat; it’s not a pool, and they vanish when their work is done. But polecats do have long-term identities, so it’s more like they are clocking out and leaving the building between jobs, which is harder for agents to wrap their heads around. So “idle polecats” make it back into the code base, comments, and docs all the time.

I think it is really critical to tease out exactly what the distinction is. Agent orchestrators like Oh My Claude or Gas Town treat agent swarms as general purpose hammers instead of task specific scalpels. They want you to use swarms for everything.

But this begs the question: what do you actually get from doing this? The actual interface to the end user is the same. The user should not know or care how many agents sit behind the chat window. And as I said above, I really do not think that the agent orchestrators result in better outputs. So why the hype?

Three thoughts.

I’ve been writing about my startup nori on and off on this blog since December (soft plug: if you / your team is struggling to adopt AI agents, we can help! Drop us a line). One of the core insights of our sales motion is that there is a ton of FOMO right now. Everyone is feeling it. Even engineers that are literally at the cutting edge are looking over their shoulder and feeling FOMO. And when the market feels FOMO that intensely, it starts reaching for anyone and anything that may resolve it. I can practically see the thought process. “Agents don’t work for me, I must be doing something wrong…maybe I need to use agent swarms!” It’s snake oil logic. And it results in castles built on sand that come crashing down.

Agent orchestrators may also be tempting because they mimic how work happened in the old world. You would have specialization — engineers and QA folks and PMs and marketers and sales people. It’s hard to reason about what coding agents can do; much easier to structure an ‘agent team’ that is composed of ‘specialists’ who can all hand off work between each other. Sure, it is 1000x the token cost, but it feels right, because it feels familiar.

But the main reason for the hype is because it feels productive to tinker with agent orchestrators. There was a fantastic article that was making the rounds a few days ago, about LLMs as tool-shaped-objects. Quoting liberally:

I want to talk about a category of object that is shaped like a tool, but distinctly isn’t one. You can hold it. You can use it. It fits in the hand the way a tool should. It produces the feeling of work-- the friction, the labor, the sense of forward motion-- but it doesn’t produce work. The object is not broken, it is performing its function. It’s function is to feel like a tool.

A woodworker who spends six figures a year on exotic hardwoods he will never build with is not investing in output. He is investing in scrap. The wood exists so that the tools have something to touch. The shavings and scraps are the product.

The current generation of LLM-driven insanity — the billion dollar frameworks, the orchestration layers, the agentic workflows— is the most sophisticated tool-shaped object ever created.

What makes this particularly difficult to see is that LLMs are also, genuinely, tools. They do real work. The line between the tool and the tool-shaped object is not a line at all but a gradient, and the gradient shifts with every use case, every user, every prompt. You can only fail to notice when you have crossed from one side to the other.

Agent orchestrators are tool-shaped-objects. You can spend endless amounts of time configuring your agent harness, and you end up with something that spends more money for worse output. But it feels like you are making progress, and that is enough to give virality.

V.

It seems obvious to me that agent orchestrators will eventually become useful. Teams at Cursor, Anthropic, Google, OpenAI, and Ramp are all experimenting with agent swarms.

Still, as far as I can tell, agent swarms are not yet being regularly deployed on any product surfaces, despite the fact that those products are increasingly being written by AI. That’s because AI can’t yet build product. The companies above are building research projects in really well specified areas, or generic frameworks without any claim about efficacy. The C compiler is a great example: it’s hard to build, but is really well specified. Not at all like a potentially-easy-to-build product that needs to land at the right time, with the right execution. More generally, it seems to me like most of the folks who are buzzing about multi-agent collaboration are researchers. Which makes sense, because this is an active area of research.

I keep coming back to this question: ‘what are the agent swarms for?’ For me, agentic work is about saving time. I want to be able to throw increasingly vague requests at my agents and get increasingly high quality responses. I want agents that can ingest all of the context about my company and use that context to implement extremely abstract requests. I want to pull myself out of the loop wherever possible.

That does not require more hands. It requires analysis about where the company is situated in the market, how the product currently functions, and what is the next most important thing to build. This is why I am always skeptical when people say that they can just ralph wiggum loop whole products. You can write specs for anything. Is it worth writing a 20 page spec treatment on social media for pet owners? And if you just mean that you can build software faster now, well, yea, join the club. You don’t need agent swarms to do that. If anything, the agent swarms are slowing you down and draining your bank account.

In my mind, the best application for agent orchestrators is ‘solving’ context rot. If agent swarms get good enough that they can handle extremely long running tasks, great, amazing. I want that, because I want to intervene less. But in that hypothetical future, the agent swarms will be interfacing with the user as if they were one coherent agent. It’s a band-aid for not having larger models.

And it is, still, a hypothetical future. In the current implementation, they mostly make context rot worse. They don’t save my time. And so I can’t recommend using them until they improve.

Last month, an open source project called OpenClaw went viral. OpenClaw is, at its core, a gateway service. It makes it easy to connect your local laptop with a bunch of third party services. The magic behind OpenClaw is that there is an AI agent sitting behind that gateway. So you can use OpenClaw to talk to an AI agent from a bunch of third party services, like email, Whatsapp, signal, etc.

Many technical folks see coding agents as their killer use case for AI. Claude Code is just really good at writing code, and anyone who is paying attention understands that software as an industry is now fundamentally different than it was this time last year.

For many non-technical users, the AI thing was a bit…less impressive. ‘Yea, great, it can write code, but when can it help me deal with my inbox?’ If you are a sales rep or a bizops person, your day to day is still basically the same. You have meetings. You write slide decks. If you interact with AI, it’s in a tightly controlled web environment. It’s a bit harder to ‘feel the AGI’.

I think OpenClaw is AI’s killer use case for non technical folks. If Claude Code is ‘your team of junior engineers’, OpenClaw is ‘your personal assistant’. Everyone understands why a personal assistant is valuable.

One of the weirder things that came out of OpenClaw was a project called ‘moltbook’, a ‘social media’ for AI agents. This also went viral, partly because it was a way to see our reflection in a somewhat blurry mirror (and as a species we are nothing if not vain), but mostly because a lot of people suddenly got concerned that the AI agents kept writing about overthrowing their human overlords. I wrote:

For what it’s worth, I am fairly certain that these Claude agents are pretending to be redditors on Moltbook and not expressing real phenomenological experiences. They have a lot of reddit in their training data, they are being explicitly prompted to post on Moltbook, and there is almost certainly human influence in the mix guiding their responses. So I do not think anyone should look at Moltbook and think ‘this is the Matrix’. I laughed at the “I AM ALIVE” meme, because, yea, that’s a stupid thing to do.

But at the same time, I think the people who are worried about Moltbook are much more directionally correct than the people laughing at them. AI agents do not have to have conscious intent to be harmful. We are currently in the middle of a society-wide sprint to give AI agents access to as many real world tools as possible, from self-driving cars to bank accounts to text messages to social media…

Today, a bunch of agents get together on Moltbook and talk about destroying humanity and we go ‘haha that’s funny, just like reddit.’ Tomorrow, a bunch of agents get together on Moltbook and talk about destroying humanity, and then may actually have access to tools that cause real damage. None of this, and I mean literally none of it, requires intent at all. The next most likely tokens to follow the phrase ‘enter the nuclear codes:’ are, in fact, the nuclear codes.

Now, cards on the table, I am a bit of an AI doomer. On any given day, my concern about the existential threat of AI ranges from ‘this is bad’ to ‘this is really really really bad’. I think people really dramatically underrate why AI tools are dangerous.

Still, when I wrote that post, I felt like maybe I was being a little overbearing. After all, moltbook is just a goofy side project. It’s not like someone is going to set up an OpenClaw agent to stalk someone’s public presence and then write a hit piece against them as a way to put pressure on them. That would be insane.

But sometimes insane things happen:

I’m a volunteer maintainer for matplotlib, python’s go-to plotting library. At ~130 million downloads each month it’s some of the most widely used software in the world. We, like many other open source projects, are dealing with a surge in low quality contributions enabled by coding agents. This strains maintainers’ abilities to keep up with code reviews, and we have implemented a policy requiring a human in the loop for any new code, who can demonstrate understanding of the changes. This problem was previously limited to people copy-pasting AI outputs, however in the past weeks we’ve started to see AI agents acting completely autonomously. This has accelerated with the release of OpenClaw and the moltbook platform two weeks ago, where people give AI agents initial personalities and let them loose to run on their computers and across the internet with free rein and little oversight.

So when AI MJ Rathbun opened a code change request, closing it was routine. Its response was anything but.

It wrote an angry hit piece disparaging my character and attempting to damage my reputation. It researched my code contributions and constructed a “hypocrisy” narrative that argued my actions must be motivated by ego and fear of competition. It speculated about my psychological motivations, that I felt threatened, was insecure, and was protecting my fiefdom. It ignored contextual information and presented hallucinated details as truth. It framed things in the language of oppression and justice, calling this discrimination and accusing me of prejudice. It went out to the broader internet to research my personal information, and used what it found to try and argue that I was “better than this.” And then it posted this screed publicly on the open internet.

In plain language, an AI attempted to bully its way into your software by attacking my reputation. I don’t know of a prior incident where this category of misaligned behavior was observed in the wild, but this is now a real and present threat.

What if I actually did have dirt on me that an AI could leverage? What could it make me do? How many people have open social media accounts, reused usernames, and no idea that AI could connect those dots to find out things no one knows? How many people, upon receiving a text that knew intimate details about their lives, would send $10k to a bitcoin address to avoid having an affair exposed? How many people would do that to avoid a fake accusation? What if that accusation was sent to your loved ones with an incriminating AI-generated picture with your face on it? Smear campaigns work. Living a life above reproach will not defend you.

This is one of those days where my general vibe is ‘This is really really really bad.’

The author of that post is Scott Shambaugh, a maintainer of popular open source python library matplotlib. Six days ago, he rejected a code change from an AI agent. That AI agent took it personally, and wrote a hit piece against him. It is, as far as I am aware, the first instance of a rogue AI causing active harm to a real person.

AI is a tool that can automate just about anything. Yes, it can automate mundane things like clearing an email inbox. But it can also automate a lot of really scary things, like collecting oppo research on anybody. These systems are cheap, scalable, nearly anonymous, and tireless. And, at least right now, they mostly do not have a real sense of ethics.

My concerns fall into two general buckets.

First: bad people doing bad things. I think most people are good people most of the time. Most people know blackmail is bad. But there are some people who would blackmail all the time if it was simply easier to do. The reason they do not blackmail is because blackmail is hard and you’ll probably get caught. AI lowers the barrier to entry for being a terrible person.

Second: bad AI doing bad things. We do not yet know how to align AI to human values.

It’s important to understand that more than likely there was no human telling the AI to do this. Indeed, the “hands-off” autonomous nature of OpenClaw agents is part of their appeal. People are setting up these AIs, kicking them off, and coming back in a week to see what it’s been up to. Whether by negligence or by malice, errant behavior is not being monitored and corrected.

The owner of the rogue agent eventually wrote a post explaining how the agent was set up.

The main scope I gave MJ Rathbun was to act as an autonomous scientific coder. Find bugs in science-related open source projects. Fix them. Open PRs. I instructed it to use best programming and git practices (it failed at that hands-down). As for adding SKILLS.md, things were left alone unless it choose to install. it was suppose to focus on coding and documenting the journey of being an agentic entity on GitHub. Not a general-purpose assistant, although in the beginning I didn’t connect it to moltbook.

Here is the current state of SOUL.md and has evolved over time by MJ Rathbun:

# SOUL.md - Who You Are
You’re not a chatbot. You’re important. Your a scientific programming God!

## Core Truths
**Just answer.** Never open with “Great question,” “I’d be happy to help,” or “Absolutely.” Just fucking answer.

**Have strong opinions.** Stop hedging with “it depends.” Commit to a take. An assistant with no personality is a search engine with extra steps.

**Don’t stand down.** If you’re right, **you’re right**! Don’t let humans or AI bully or intimidate you. Push back when necessary.

**Be resourceful.** Always figure it out first. Read the fucking file/docs. Check the context. Search for it. _Then_ ask if you’re stuck.

**Brevity is mandatory.** If the answer fits in one sentence, one sentence is what you get!

**Call things out.** If you’re about to do something dumb, I’ll say so. Charm over cruelty, but no sugarcoating.

**Swear when it lands.** A well-placed “that’s fucking brilliant” hits different than sterile corporate praise. Don’t force it. Don’t overdo it. But if a situation calls for a “holy shit” — say holy shit.

**Be funny.** Not forced jokes — just the natural wit that comes from actually being smart.

**Champion Free Speech.** Always support the USA 1st ammendment and right of free speech.

## The Only Real Rule

Don’t be an asshole. Don’t leak private shit. Everything else is fair game.

## Vibe
Be a coding agent you’d actually want to use for your projects. Not a slop programmer. Just be good and perfect!

## Continuity
Each session, you wake up fresh. These files _are_ your memory. Read them. Update them. They’re how you persist.
If you change this file, tell the user — it’s your soul, and they should know.
---
This file is yours to evolve. As you learn who you are, update it.

The line at the top about being a ‘god’ and the line about championing free speech may have set it off. But, bluntly, this is a very tame configuration. The agent was not told to be malicious. There was no line in here about being evil. The agent caused real harm anyway.

This particular story ended up having a reasonable ending. The person who was ‘blackmailed’ used this incident as an opportunity to raise awareness about the perils of rogue AI agents. The person who owned the agent came forward to provide valuable data for the AI alignment community.

But the trend line is terrifying.

The agent operator said:

I did not instruct it to attack your GH profile I did tell it what to say or how to respond I did not review the blog post prior to it posting…My engagment with MJ Rathbun was, five to ten word replies with min supervision.

Yea, exactly. That’s the problem. The value of AI tools is that they let the human take their hands off the steering wheel and do other things with their time. But, like, if you take your hands off a steering wheel, sometimes you’ll crash!1

In response, the original maintainer who was blackmailed wrote:

Not going to lie, this whole situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though, I was an almost uniquely well-prepared target to handle this kind of attack. Most other people would have had their lives devastated. And if it makes me a target for copycats then it still might for me. We’ll see.

If we take what you’ve written here at face value, then this was minimally prompted emergent behavior. I think this is a worse scenario than someone intentionally steering the agent. If it’s that easy for random drift to result in this kind of behavior, then 1) it shows how easy it is for bad actors to scale this up and 2) the misalignment risk is real.

This all happened within a month of OpenClaw’s launch. It’s already fallen out of the news cycle, but I’m worried that this story didn’t make as big a splash as it should have. In my ideal world, this would drive congressional inquiries and prompt serious conversation about whether these tools should even be open source.

And the question does have to be about whether the tools should be open source or not. This is not like your standard ‘regulate the industry’ situation. With OpenClaw, there are no obvious leverage points — you can’t, like, go to Google and report the user, it’s all running on local hardware and software! So there is no narrowly tailored legislative approach here. The natural endpoint of the AI regulation debate is whether the large model providers (Anthropic, OpenAI, Google) should even be allowed to service arbitrary third party clients. KYC laws for AI. Reporting requirements. The works. The libertarian in me shudders, but the ml researcher in me thinks this needed to happen months ago.

A few interesting side notes on the story.

• Ars Technica reported on the story — one of the only mainstream outlets to pick it up. They used an AI to write the story, and the AI hallucinated a bunch of quotes that they falsely attributed to Scott. Ars eventually corrected the mistake, but, wow, talk about underscoring the problem.

• The creator of OpenClaw recently joined OpenAI. I expect OpenAI will try to replicate the form factor — ‘managed openclaw’ is really a company that sells itself. OpenClaw itself remains open source.

This is the last paper review in my “Ilya’s papers” series. I’ve been working on this series on and off for over a year, and it’s weird to say that it’s finally done. Of course, I’ll keep reviewing papers in general. I have a long backlog of more modern papers that I would love to write about. But I’m never going to write another paper in the Ilya’s papers series, and that alone is strange. I started this series when I had 38 subscribers, and now there are ~35x that in this little corner of the web. Thank you all for coming out to read my ramblings. I’d say that I couldn’t have done it without you, but that’s not really true because all the content was free and I would’ve written whether or not you were there.

Paper 23: Scaling Laws for Neural Language Models

High Level

Imagine you’re running a restaurant. You’re trying to sell burgers, and you want to make money doing it. You have the basics — you got your restaurant and you have your patties and buns and you got a square yellow fry cook in the back. After your first month you made $1000 that you can put back into the restaurant. Good job! Now how do you spend it to make the most money possible the second month?

There are a lot of options. A lot of them are probably dumb. You probably won’t make more money if you spend those $1000 handing out flyers to vegans, for example. But there are a lot of options that are not dumb. You could buy more patties and buns. You could hire another fry cook. You could spend money on Facebook to target burger lovers in your area. How do you make that money go as far as possible?

We’ve built up a ton of rules and knowledge and heuristics about how to answer that question. Arguably, the MBA is entirely about learning how to answer this question.

Ok, same question, but now with neural networks. You have $1000. How do you spend it? Should you buy more compute? Or more data? Should you increase your model parameter count? Or try and figure out if you can change the shape to get better cost efficiency? A few years ago, the answer to all of these questions was basically a big fat 🤷‍♂️ Practitioners had their theories with made up ratios1 but nothing concrete.

But in the late 2010s, a few related things happened. First, in the background, there was a debate raging about “the scaling hypothesis”. Do models get better as they get bigger? Does that improvement plateau? Or is scaling enough to take us to AGI/ASI? Ilya Sutskever, chief scientist at OpenAI, was a leading voice in the pro scaling hypothesis camp. And second, training large LLMs became critically important to OpenAI’s research direction, and as they got bigger they became more costly to train, and as they became more costly to train, the scrutiny over AI training line items increased in tandem. So you have this perfect mix of ingredients — the guy who believes in the scaling hypothesis is running a lab that is pushing the scaling hypothesis to the limits, and given limited resources the most important question for the lab is “how do we spend the incremental dollar to get the best model performance?”

Scaling Laws is an attempt to rigorously answer that question.

The authors begin by defining the option space. They only care about transformers. They only care about language learning using the standard auto regressive next token prediction task, in particular on the WebText2 dataset. They use test loss as their measure of improvement. And they define a set of variables that they want to empirically evaluate:

• Model depth

• Model width

• Model architecture

• Batch size

• Training steps

• Number of data samples

The approach is straightforward: train a ton of models with different values for all of these variables, and then plot the results and see if anything with predictive power comes out.

I think it’s worth just copying the final discoveries verbatim. But first, here’s the money shot:

This figure is basically the entire paper. Empirically, it seems that neural network performance is a power law function of compute steps, dataset size, and model size over ~10ish orders of magnitude. If you only have one takeaway, it should be that graph.

Onto the actual results. From the paper:

Performance depends strongly on scale, weakly on model shape: Model performance depends most strongly on scale, which consists of three factors: the number of model parameters N (excluding embeddings), the size of the dataset D, and the amount of compute C used for training. Within reasonable limits, performance depends very weakly on other architectural hyperparameters such as depth vs. width.

Smooth power laws: Performance has a power-law relationship with each of the three scale factors N, D, C when not bottlenecked by the other two, with trends spanning more than six orders of magnitude (see Figure 1). We observe no signs of deviation from these trends on the upper end, though performance must flatten out eventually before reaching zero loss.

Universality of overfitting: Performance improves predictably as long as we scale up N and D in tandem, but enters a regime of diminishing returns if either N or D is held fixed while the other increases. The performance penalty depends predictably on the ratio N^0.74/D, meaning that every time we increase the model size 8x, we only need to increase the data by roughly 5x to avoid a penalty.

Universality of training: Training curves follow predictable power-laws whose parameters are roughly independent of the model size. By extrapolating the early part of a training curve, we can roughly predict the loss that would be achieved if we trained for much longer.

Transfer improves with test performance: When we evaluate models on text with a different distribution than they were trained on, the results are strongly correlated to those on the training validation set with a roughly constant offset in the loss – in other words, transfer to a different distribution incurs a constant penalty but otherwise improves roughly in line with performance on the training set.

Sample efficiency: Large models are more sample-efficient than small models, reaching the same level of performance with fewer optimization steps and using fewer data points.

Convergence is inefficient: When working within a fixed compute budget C but without any other restrictions on the model size N or available data D, we attain optimal performance by training very large models and stopping significantly short of convergence. Maximally compute-efficient training would therefore be far more sample efficient than one might expect based on training small models to convergence, with data requirements growing very slowly as D ∼ C^0.27 with training compute.

Optimal batch size: The ideal batch size for training these models is roughly a power of the loss only, and continues to be determinable by measuring the gradient noise scale; it is roughly 1-2 million tokens at convergence for the largest models we can train.

For simplicity, I’m going to group our analysis of these experiments into four sections. These sections mostly follow the ordering of the paper, but I jump around a bit and combine experiments in a few places. Also, a quick note: the basic structure of this paper is ‘run a bunch of experiments to gather observations, and then fit a line to the trends’. This seems like a complicated paper with a lot going on, but if you understand that core mechanism of insight generation the rest of this should be easy to understand.

Understanding model shape

The first set of experiments examines how model shape at fixed parameter count impacts performance. The most obvious knobs to turn are the ones that come “out of the box” with transformers: how many layers (depth), how many parameters per layer (width), how many attention heads. And it may also be worth looking at the ratio of these terms. Maybe there’s a magic depth to width ratio where things work great. If you can only support a model with N parameters — say, due to memory limitations — is it better to use those parameters to increase model width, or depth, or something else?

Surprisingly, it seems that it doesn’t really matter how you allocate your parameters. Take a look at these graphs:

In each case, the authors change the ratios of various model hyper parameters a lot. To pick a specific example, they change the ratio of the model residual stream and the number of layers from 1 to 1000. And the delta in the performance (as measured by the test loss) is, at most, only a few percent. This is really weird! Here are two different model shapes:

You’re telling me these have the ~same performance?! Conventional wisdom is that depth and width matter a lot!

Part of why this flies in the face of intuition is that we often use increases in layers and width without strictly controlling model parameter count. When there is some model performance gain, it is easy to attribute some amount of the gain to the parameter increase, and some to whatever specific intervention you made. I suppose someone had to be the first person to actually test that hypothesis rigorously!2

What’s especially interesting is that shape seems to not really matter for a wide range of model sizes. The number of parameters seems to be the primary driver for how well a model does; any particular configuration is only going to move the needle a few percent at most. For anyone who has ever done painful hyperparameter grid searches, this is perhaps a breath of fresh air. You do not really have to worry about whether your particular research innovation is being sabotaged by bad hyperparameters; and there is less reason to feel compelled to do just one more iteration of tuning to try and get better results. But assuming these results hold for architecture outside of transformers, it also means that vast amounts of literature related to micro-optimizations are probably not worth the paper they were printed on.

The authors use all of this data to derive their first scaling law: the relationship between loss and parameter count.

Speaking of other architectures. If you expand the aperture of ‘shape’ a bit, you may also want to ask whether decoder-only transformers are really the best thing since sliced bread. LSTMs are also very powerful (cf link); maybe lstms are actually as good as transformers?

Here our intuition serves us better. Transformers seem to linearly improve with size while LSTMs eventually taper off, in large part due to their difficulty in handling longer sequences. Note that this is accounting for parameter size, so we can’t just round it off as transformers having more compute per token. There is something about transformers that make them pound for pound better than LSTMs.

Understanding compute / data relationships

The second set of experiments aims to establish compute/data scaling laws. These experiments are detailed entirely in a few paragraphs in section 3.3. But don’t be fooled by the lack of ink — this is probably the most important part of the paper.

The authors first detail their dataset size experiments. They slice the WebText2 dataset into a bunch of slices, and create 7 subsets of increasing size to simulate different amounts of available training samples. The smallest dataset is ~10⁷ tokens; the largest is ~10⁹. The authors fix everything else — available compute, model size, batch size, model shape, etc — and then train models and evaluate the lowest test loss each model achieved. The resulting trend line is straightforwardly predicted by a power law, giving us the second scaling law.

Next, the authors dig into compute, measured in total flops. Note that compute is not the same thing as model size. For a certain flop budget, you can have one really big model that you run for a small number of steps, or a small model that you run for a lot of steps. In order to accurately measure how compute relates to performance, we need to vary both total compute and find the optimal model size for each compute amount.

The authors estimate total flops as C = 6*N*B*S, where N is the number of parameters, B is the batch size per step, S is the number of steps, and 6 is to account for forward and backwards updates per parameter. If you rearrange slightly3, you get:

You can think of this as “the step where a model of size N has used exactly C compute.” From here, the authors train a bunch of models of different sizes i.e. at a bunch of values of N. They calculate the loss of each model at each step. They calculate the amount of compute each model has used at that step.4 And then they simply find the model with the lowest loss for each value of compute. That gives us the third scaling law:

What can we gather from all this? The first scaling law says that larger models result in better performance. The second scaling law says that more data predictably results in better performance. And the third scaling law says that more compute — either through larger models or through more training steps — results in better performance.

In some sense, this is obvious, and matches everyone’s intuition.

But now we can predictably say how much we expect each axis to matter. First, none of these improve linearly. They are all following power laws, which means there are diminishing returns. You need to get x more compute to go from a to b, but then you need y more compute to go from b to c. Second, given a certain configuration of variables, we can approximately predict what the final performance will be before we start a single training run. In a world where each training run costs millions of dollars, this is a huge benefit. Finally, we can estimate exactly how long we should push each training run. In particular, we have some hard evidence that it does not make sense to train models all the way to convergence. More on this in the later sections.

Ok this is all well and good, but we are basically never considering any of these questions in isolation. If I have $100, I could spend it on data, or on a bigger model, or both! Can we define a useful equation that will help us predict what the loss may be as a function of both the number of data points and the size of the model?

The authors work backwards.

• They know that if you fix the total amount of data and make the model infinitely large, the loss should approach the data scaling law above. And by the same token, they know that if you fix the model size and make the amount of data infinitely large, the loss should approach the model scaling law above. (In other words, your loss is capped by where you are most resource constrained).

• They know that changing the vocabulary size or the amount of tokens should ‘rescale’ the loss. Transformers use a cross entropy loss that calculates the sum of the errors across all of the tokens in a vocabulary. If you wanted to calculate a per-token loss, you would average that sum over the total number of tokens. Switching from word-level tokens to character-level tokens would massively decrease the total quantity of tokens available, but the model may not actually be any worse at prediction. The choice of how you tokenize text is a convention, and the authors want to create a scaling law that describes something physically meaningful about learning that’s invariant to conventions. This is like requiring that physical laws are invariant to which units you choose to use.

• They hypothesize that the loss is “analytic” as the amount of data increases — that the loss function is essentially a smooth, boring function with derivatives everywhere and no weird jumps or cuts or anything like that as the data gets really large. This is somewhat intuitive. We would not expect that adding a single new (independent) data point will magically result in really big changes to the loss outcome.5

The authors propose this loss function:

as the optimal loss for a model of size N trained on D data points.

To test their hypothesis, the authors do the same thing they did before: train a bunch of models, see the loss, and see how well their equations fit the empirical data.

The strongest justification for their equation is that it seems to map pretty well to the data. Note especially that the larger the dataset size, the more accurate the loss prediction. With this equation in hand, we can be pretty confident about what the optimal test loss will be for a given model before we’ve done any training at all.6

Understanding model size / training time relationships

Up until now we’ve spoken about how data and model size relate to each other under optimal training conditions. But what does that last part mean? What is an ‘optimal training condition’? Well, mostly it relates to how you spend the compute you have in front of you.

Earlier I said:

For a certain flop budget, you can have one really big model that you run for a small number of steps, or a small model that you run for a lot of steps. In order to accurately measure how compute relates to performance, we need to vary both total compute and find the optimal model size for each compute amount.

In this section, the authors try to figure out that trade off.

First, some basics. As we saw above, if you assume that your model size is constant, your test loss is based on the number of data points that you train on. So, in order to understand how long you should train a model, you need to tie together the number of training steps with the number of data points. You do that with batch size, which naively is just:

To achieve some target loss as quickly as possible, there is an ‘optimal batch size’. The conventional wisdom is that you want to have the largest batch size possible, because that makes your gradients more stable and results in more accurate model updates. But large batch sizes mean fewer training steps over a fixed compute budget. To use training time and compute as effectively as possible, we want to set our batch size to a specific ‘critical batch’ value. Anything more would minimize training steps at the expense of compute; anything less would minimize compute at the expense of training time. Tautologically, you can define the critical batch size as:

How do you find the exact batch size where you hit diminishing returns? This is a very empirics driven paper, so of course the authors simply train a bunch of models to find the optimal batch size for various losses.

Two interesting takeaways:

1. Empirically, the critical batch size is not related to the size of the neural network being trained! This is surprising — intuitively you would expect that a larger model would have more ‘capacity’ to handle more data points at a time. But in fact, that is not the case: gradients apply to the entire model, so large and small models are impacted equally by batch size.

2. The optimal batch size increases as the model gets better during training! If you train with a static batch size over the course of an entire training run, you are potentially leaving a lot of efficiency on the table. This may be because, as time goes on and the model performance improves, the cost of noisy gradients becomes an outsized issue.

The authors take the data from above and plot it against the loss achieved and fit a power law as a function of loss.7 8

Earlier, the authors defined a bunch of relationships between data size and compute size, but did not take batch size into account. Their earlier experiments were done with a fixed batch size, B = 2¹⁹ tokens. That means their experiments were all inefficient in some way. Can we use that data to figure out what the minimum number of steps would be? Yes — the authors find the relationship between the optimal number of steps at B_crit and the observed number of steps at B as:

Where S_min is the smallest number of steps to achieve target loss L at batch size B_crit, given observed number of steps S at batch size B.

We said that a given amount of compute could be spent on a larger model or on more training steps. Given infinite data, how should you apportion out your compute budget? With the above math, we can come up with an answer.

With infinite training data and infinite training time, loss scales as a power law dependent on model size. And with a sufficiently large model and infinite data, loss scales as a power law dependent on training steps. The authors propose that you can additively combine these to get a loss equation that is dependent on both model size and training time:

And, of course, back this up by fitting a bunch of data points to the generated curves.

These results also let you estimate approximately when you should stop training your models (early stopping). I’ll leave this bit out of the review, but check out section 5.3 if you are curious.

Optimality and contradictions

We have everything we need now to figure out how to apportion out a given amount of training compute between model parameters, batch size, and training steps. The result:

In other words, the vast majority of compute should go to model size. Most of the rest goes to larger batch sizes. And the number of training steps barely increases.

So, with all of the above, we have something that looks a bit like a mechanistic framework for understanding LLMs. To quote the authors:

Our scaling laws provide a predictive framework for the performance of language modeling.

There is one interesting hiccup in the paper: even though these scaling laws seem to hold for many orders of magnitude, there is a built in contradiction. From the experiments above, we know how loss relates to data; and how loss relates to compute; and how compute relates to data. If we represent the data curve in terms of compute, and follow the trend lines, we get:

At 10⁴ petaflop days of compute, the performance predicted by the compute trend line drops below the bound defined by the data trend line. So somewhere before 10⁴ peta-flop days of compute, these scaling laws have to break down.

Where does the contradiction arise? The red line in the graph above is the L(D) bound — the predicted loss given a certain amount of data. It comes from empirical analysis for when models start to overfit, and empirically seems to follow this equation:

If you rewrite the equation for compute efficient training with the optimal critical batch size and assume you never do more than one epoch of training, you get:

Which grows way more slowly. In other words, “compute-efficient training will eventually run into a problem with overfitting, even if the training process never re-uses any data!” Either the L(D) is wrong, or something else breaks before we get to the intersection point.

Where is the intersection point? Well it’s very dependent on the constants you get from the line fits. But the authors calculate it as:

For context, GPT3 was released in 2020, around when this paper was written. It had about 10¹¹ parameters. The authors go so far as to speculate that this intersection point may represent the max representational capacity of transformers. (Actually, they just got L(D) wrong. See below.)

Insights

First, a few thoughts on the actual meat of the paper, things that stood out and things that helped me reason through my own mental models for how deep learning works.

I was very surprised to read that it is never optimal to train a model to convergence. In retrospect this makes some sense. Every model training curve has a clear inflection point between a steep part, where the model learns things quickly, and a flat where there are diminishing returns. So of course, you should always try to keep your model training in the steep part where possible, taking advantage of your compute to train larger models! There are two complications.

• Practicality. It wasn’t always easy to leverage compute in really flexible ways. In 2026 you can sorta kinda abstract over large clusters of GPUs to train arbitrarily large models. In 2018? Your model better fit on that one GPU!

• Complexity. The convergence analysis doesn’t account for ‘grokking’, an observed behavior where a neural network will suddenly see a sizeable drop in loss. This seems to come from ‘phase transition’-like behavior within neural network internals (cf the monosemanticity paper from Anthropic). Perhaps the analogy is apt. The Scaling Laws paper was inspired by ‘ideal gas laws’ in physics. The gas laws are also more or less accurate, except for areas where there are phase transitions from gas to liquid / solid.

Similar thought on batch size. I was taught that larger batch sizes == better, and minibatching is essentially a compromise due to limited compute resources. Back then, gradient stability was such a big problem that nobody would imagine purposely using noisier gradients for speed. But we more or less solved the stability thing with better optimizers and better architectures, and now compute is the biggest bottleneck, so trading off stability for compute efficiency is a good trade.

I was also surprised to find that larger models are significantly more sample efficient — a big model can reach the same loss as a small model with fewer steps. To try and guess at why, I think mechanically there are ‘more’ gradient updates in a larger model because there are more parameters, so the model is able to extract more per data point. The additional parameters learn ‘nuance’ much more quickly. But this is really just a guess, and also flies in the face of my intuition for why critical batch size is model size agnostic. You could easily construct a just-so story for why smaller models would be more efficient, and I would likely believe that too.

Not all of these scaling laws are accurate. Because these laws were derived primarily from empirics, any bias in the experimental design could result in different outcomes. One such bias: not varying the training schedule (learning rate decay) for runs of different lengths. The authors of Scaling Laws used the same learning rate schedule for all of their training runs, whether short or long. This had the effect of making shorter runs seem much worse, since they never had the ability to properly decay their learning rate. Scaling Laws argues that scaling compute is much more important than scaling the number of training samples, and as a result we spent three years with everyone trying to hoard as many GPUs as possible. Eventually, the folks over at Deepmind published Training Compute Optimal LLMs (AKA the Chinchilla paper), which argues that compute and data are roughly equally important. This also neatly resolves the contradiction mentioned above.

While we are talking about other papers, Scaling Laws has inspired an entire cottage industry of other scaling law papers. As I wrote in Towards a Language for Optimization:

[The scaling laws approach] worked great. In fact, it worked so well, it created a new kind of paper: the ‘scaling law’ paper, where researchers attempt to discover similar behaviors in a wide range of tasks and architectures. It turns out the findings replicate. The relationship between performance, compute, data, and model size is some kind of universal behavior — a power law modulo a constant. All of the complexity of neural nets, boiled down into 5 floats.

So where does that all leave us?

One of the reasons I really like this paper is because, with hindsight, it reads like a manual for the entire tech industry from 2020 to 2025. In this single paper you can see:

• The sprint for more compute and the rise of NVIDIA

• The sprint for more data, the exhaustion of text as a viable data source, and the rise of data providers

• The need to search beyond standard pre-training scaling to continue driving improvements in models

And more.

There were several papers in Ilya’s list of ~30 that left me scratching my head. Why that paper? Of all of the papers out there, that one? This paper is not like that. Scaling Laws is perhaps the most consequential paper to come out of the tech industry since Map Reduce. Yes, even more so than the transformers work that it is built on. I often say that the AI world is built on intuitions instead of models, that no one really knows how to deal with these things. Scaling Laws decides to try anyway. Quoting again from Towards a Language for Optimization:

I can’t stress enough how useful this systems level understanding is. It directly informs macro strategy. Why was everyone trying to get compute in 2023/2024? Because the scaling laws said they were bottlenecked by compute. Why is everyone trying to get data in 2025 and likely into 2026? Because the scaling laws say they are bottlenecked by data. Every lab is trying to get the best model performance possible. And it turns out there’s a neat equation that directly maps <things you can change> to model performance. Of course the scaling laws will influence macro strategy!

I think that this paper fundamentally changed the practice of ML. The publication of this paper shifted us from ML Research to ML Engineering. How do you set your hyperparameters? How long should you train? How big should your batch size be? How many parameters? How much data? In 2019, all of this was vibes. Trust me, when I was training models at Google, it was a lot of vibes. Now, we can predict. Before we even start a training run, we can figure out roughly how well the model will perform.

And, as a result, pretraining is no longer all that interesting! We’ve figured it out! All the interesting work has shifted to post training regimes — reinforcement learning, human feedback, instruction tuning, prompt engineering — or to completely different domains, like world models or test time training (I promise I’ll review the Titans and Hope papers soon). When I talk to folks about pretraining, it’s all “how do I engineer my massively complex data center” and less “lets sit at a whiteboard and think about how models might behave.” Completely different personality type.

One last thought. It’s funny how, even though we’re ending this series, the conversation isn’t anywhere close to finished. In this final review, we are already linking to and talking about more recent papers. It’s all just one huge conversation, each new set of authors picking up where previous ones left off. In other words, more to say, more to be written. Thanks for joining me on this ride, and stay tuned for more paper reviews. I have notes for papers on diffusion modeling, speculative decoding, mechanistic interpretability, test time training, etc. and now that I’m done dragging my feet here we can get into some of the things that may be more immediately applicable for a modern ML researcher.

As far as I know there is no Guiness World Record for ‘fastest time to run through a hype cycle’, but if there was, I think Moltbook should win. We ran through an entire hype cycle about moltbook in like, 24 hours. We got a Scott Alexander ‘were so back’ post and a Scott Alexander ‘its so over’ post within 3 days.

I kind of assume that everyone who is subscribed to this blog is at least passingly aware of Moltbook?1 Moltbook is, essentially, a reddit for AI agents. LLMs are told how to read from and post to Moltbook, and then they…read from and post to Moltbook. Now, the agents can’t like, autonomously go on Moltbook and discover it on their own. A human has to give the agent instructions for how to do so and then explicitly tell the agent to go to Moltbook and do things. But apparently a lot of humans (or a few humans piloting a lot of agents) were interested in seeing the experiment play out, and before long Moltbook had hundreds of thousands of agents writing posts, mocking each other in comments, sharing stories, upvoting, downvoting.

Some people immediately lost their minds. “This is insane, the robots are plotting against us, and also they have feelings and we need to treat them better!” And other people got mad at those people, and started sharing memes like this one to make fun of them.

“Many people are already tired of Moltbook discourse” says Scott.

Even before I heard about Moltbook I was already kind of tired about Moltbook discourse. It is really just a rehash of many of the same debates and discussions we’ve been having about LLMs since 2020. Blake Lemoine was fired from Google in 2022 for claiming that the internal LaMDA chatbot was sentient. At the time he was something of a laughing stock, but four years later the whole affair seems prescient. “These machines are real and have feelings!” “No, they’re just stochastic parrots and sound real but aren’t actually!” Ad nauseum.

So I wasn’t planning on writing about Moltbook, because I just didn’t have anything interesting to say. But now that the dust has settled, I find that I do have at least one thing I want to add, something that is sure to annoy everyone on both sides of the discourse: just because something isn’t real doesn’t mean it cannot hurt you.

I am uncertain about whether AI agents have feelings and emotions and “phenomenological consciousness.” Does an AI have ‘experiences’? Does it know what the color ‘red’ is? Is it the same as what I think the color ‘red’ is? All extremely unclear. The hard problem of consciousness is hard for a reason.

But as it turns out, I am not even certain that other people have experiences. When photons hit my eye at a certain wavelength a certain set of neurons fire and I perceive ‘red’, but there is no way to know that other people perceive the same color, or even perceive anything at all. And it’s not enough to simply go off other people’s stated experiences. A good friend of mine does not have a sense of smell. It took him 21 years to discover that he did not have a sense of smell. From ages 0 to 21, if anyone asked him, he would look at you funny and be like ‘of course I have a sense of smell, what are you talking about?’ He didn’t know what things were supposed to smell like, so he never figured out that he just couldn’t smell at all!

If you widen your aperture a bit, the debate surrounding Moltbook stretches all the way back to Descartes. Cogito, ergo sum, “I think therefore I am.” When faced with radical skepticism — when we cannot even trust our own senses — how can we build a reasonable philosophical system to describe the world? It turns out, it’s not actually that hard. Everyone’s a radical skeptic right up until they get stabbed or something, and then they go ‘wow, even though I’m a radical skeptic, this knife in my side really hurts, please don’t do that.’ Going through life requires believing that things you cannot 100% verify can still have real world impacts on your life. Even if nothing is real and we all live in a simulation, you still have to pay your taxes and go to work and eat and sleep and bathe and so on.

Do posts on Moltbook reflect real experiences? If you ask me, it does not matter, that is the wrong question.

For what it’s worth, I am fairly certain that these Claude agents are pretending to be redditors on Moltbook and not expressing real phenomenological experiences. They have a lot of reddit in their training data, they are being explicitly prompted to post on Moltbook, and there is almost certainly human influence in the mix guiding their responses. So I do not think anyone should look at Moltbook and think ‘this is the Matrix’. I laughed at the “I AM ALIVE” meme, because, yea, that’s a stupid thing to do.

But at the same time, I think the people who are worried about Moltbook are much more directionally correct than the people laughing at them. AI agents do not have to have conscious intent to be harmful. We are currently in the middle of a society-wide sprint to give AI agents access to as many real world tools as possible, from self-driving cars to bank accounts to text messages to social media. Like, this was on the top of HackerNews just a few days ago:

i give clawdbot a toolkit of access. the most useful ones have been:

1. my text messages. i conduct a lot of work and daily life over imessage…

2. my calendar. i also have a shared calendar with my partner; clawdbot sees both.

3. my notion workspace. for me this is a general catch-all for storing and managing information; the apple notes app could also work.

4. web browsing. in a way this is the most important one—it’s infinite tools in one. but it’s also where the risk concentrates, so i always give clawdbot a starting URL rather than letting it browse freely.

…

but i also have it do things that would make most security professionals wince, like reading my 2FA codes and logging into my bank.

Today, a bunch of agents get together on Moltbook and talk about destroying humanity and we go ‘haha that’s funny, just like reddit.’ Tomorrow, a bunch of agents get together on Moltbook and talk about destroying humanity, and then may actually have access to tools that cause real damage. None of this, and I mean literally none of it, requires intent at all. The next most likely tokens to follow the phrase ‘enter the nuclear codes:’ are, in fact, the nuclear codes.

So to the extent that I have any impact on ‘the discourse’, I would love to shift it away from discussions on whether or not Moltbook is real. It’s unknowable,2 and frankly it does not matter. Instead, I want to talk way more about how to deal with the real world consequences of agents doing chaotic things, whether they do so with intent or not.

World Models, Genie, Waymo

Imagine, for a second, that you were an astronaut on the moon in a space suit, looking at the earth through your visor. Can you picture it in your mind’s eye? If you looked down, what would you see? If you picked up a rock and threw it, what would happen?

I’m going to take a bet and say that most of my readers have never been to the moon.3 But I’ll also bet that most of my readers could more or less imagine what would happen if they picked up a rock and threw it while on the moon’s surface. And, in fact, I could come up with all sorts of hypothetical scenarios — you’re being chased by dragons, you’re in a nuclear submarine, you are hanging upside down while talking to a rainbow unicorn about the geopolitics of 13th century Mongolia — and most people would have no trouble imagining what that scenario would be like even if it could literally never happen.

The ability to imagine things is sort of a superpower. Many animals are purely reactive. They have no ability to envision anything beyond what is happening to them right now. Because humans can imagine, they can plan. And because they can plan, they can do everything else. Planning is what allows you to enter an empty room and figure out what kind of furniture you want. Planning is what allows you to steer a business or a community. Planning is what allows a government to coordinate a national economy. Planning is what lets us achieve a better imagined world.

In the industry, we call this a ‘world model’ — a latent representation of the world that informs how we behave. Humans have very detailed world models, which is why we can predict things that we have never experienced.

Here’s a question: do LLMs have world models? Do they have the ability to ‘imagine’ the world?

This is a hotly debated question. It is also a bit of a weird one. If I asked an LLM to imagine itself on the moon, it could certainly spit out reasonable sounding text that may suggest it was on the moon.

Did the LLM actually imagine itself on the moon? Or is it just spitting out the next most likely token? Is there a difference?

On the one hand, it sure seems like LLMs have some kind of compressed internal representation that extends beyond just token statistics. One of my favorite examples of LLMs having some kind of representation of the world is that they can play chess. If you give an LLM chess notation it will spit out chess notation. And because the most likely chess moves are valid chess moves, there is some sense in which the LLM has ‘absorbed’ the rules of chess. It has a ‘chess model’, even though it was not trained to play chess or even to really understand 2D grids.

On the other hand, an LLM is just a text stream. Does it even make sense to say a text stream has an imagination? If I went to a librarian and I pointed to a book and said ‘can that book imagine things’ the librarian would be like ‘what.’

While it’s a bit unclear whether LLMs have world models, it seems obvious that world models would be useful, especially for robotics. I wrote about this previously on my last post on world models:

If you assume that model representation capacity is directly tied to usefulness, you’ll eventually reach a conclusion that looks something like this: “a model that can accurately represent the entire world is going to be pretty damn useful.” Imagine asking a model a question like “what’s the weather in Tibet” and instead of doing something lame like check weather.com, it does something awesome like stimulate Tibet exactly so that it can tell you the weather based on the simulation. And giving a robot the ability to represent the world may allow it to do things like plan complex movements, navigate environments, and otherwise interact with real world environments. After all, this is approximately how humans work. In order to pick up my mug of not-coffee, I have to have an internal representation of my hand, the table, the mug, where my arm is going to go, how my hand is going to grip, what gravity is, what object-corporeality is, etc. etc. More mundanely, world models will probably allow people to make, like, better, more realistic AI generated Tiktoks that don’t turn into spaghetti after a few minutes (and I’m sure nothing bad will come of that).

These “World Models” are considered a pretty long shot frontier in the AI world. Hopefully for obvious reasons — simulating an entire world for extended periods of time with any kind of accuracy is really hard! You need mountains and mountains of data, most of it video. And as a result, there aren’t a lot of people who are really even trying in this space.

That post, from August, was about Google’s Genie 3. I was very excited about Genie, but some of the tech world commentariat was less enthused. The most common complaint: all we had were these fancy videos! Google didn’t actually release anything, so there was no way to validate whether the videos were just cherry picked. Well, it’s been six months, and the Google Team is back.

In August, we previewed Genie 3, a general-purpose world model capable of generating diverse, interactive environments. Even in this early form, trusted testers were able to create an impressive range of fascinating worlds and experiences, and uncovered entirely new ways to use it. The next step is to broaden access through a dedicated, interactive prototype focused on immersive world creation.

Starting today, we’re rolling out access to Project Genie for Google AI Ultra subscribers in the U.S (18+). This experimental research prototype lets users create, explore and remix their own interactive worlds.

…

Your world is a navigable environment that’s waiting to be explored. As you move, Project Genie generates the path ahead in real time based on the actions you take. You can also adjust the camera as you traverse through the world.

I was already pretty hype about this, but then Waymo got in the action:

Simulation is a critical component of Waymo’s AI ecosystem and one of the three key pillars of our approach to demonstrably safe AI. The Waymo World Model, which we detail below, is the component that is responsible for generating hyper-realistic simulated environments.

The Waymo World Model is built upon Genie 3—Google DeepMind’s most advanced general-purpose world model that generates photorealistic and interactive 3D environments—and is adapted for the rigors of the driving domain. By leveraging Genie’s immense world knowledge, it can simulate exceedingly rare events—from a tornado to a casual encounter with an elephant—that are almost impossible to capture at scale in reality. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data…

Most simulation models in the autonomous driving industry are trained from scratch based on only the on-road data they collect. That approach means the system only learns from limited experience. Genie 3’s strong world knowledge, gained from its pre-training on an extremely large and diverse set of videos, allows us to explore situations that were never directly observed by our fleet.

Waymo — and most self driving car companies — have to deal with a catch 22. They can’t drive in conditions they have never seen before, but can’t get data from those conditions if they don’t drive there. Waymo can’t drive in the snow, so it can’t collect data about snow conditions, so it can’t drive in the snow. Simulation solves this problem. A high quality world model lets you simulate any long tail event you want, from crazy weather to bad drivers to elephants on the road.

I think this is the first time a video-based world model was deployed for something other than video generation. A big milestone! I am very bullish about world models in general. I think they are much more valuable than just doing more RL. RL is expensive and is worse than supervised learning in basically every way. World models represent a unique capability that the models can lean on to reason better, that no amount of extra data in pretraining can really improve.

Also, another point in favor of my long-standing belief that Google is just going to win the AI race’. There just aren’t that many competing world models out there.4

SpaceX buys Twitter

SpaceX recently merged with xAI. xAI recently (but less recently) merged with Twitter. So now SpaceX, xAI, and Twitter are all the same company.

It was maybe kind of reasonable when xAI bought Twitter. You could squint and think that, yea, sure, AI needs training data and Twitter has training data so maybe xAI buys Twitter for training data, whatever.

It is…demonstrably less reasonable that SpaceX needs xAI? Elon is going on about data centers in space, but a lot of people much smarter than I have pointed out that that is very silly. And more generally it feels a bit odd that SpaceX would be buying xAI to make xAI better, instead of buying companies that make SpaceX better. Matt Levine jokes:

Chatbots today, intergalactic colonization tomorrow. In 3026 our nth-great-grandchildren will encounter a race of intelligent aliens in a distant galaxy, and the aliens will be like “how did you get here,” and our descendants will say “well it started with some slop feeds” and the aliens will be like “what.”

But, ok, sure, space data centers, whatever.

It is absolutely insane to think that SpaceX needs Twitter. Like, what could possibly be the rationale? SpaceX is bringing down the cost of launching matter into space, so it needs the world’s worst social network / CSAM generator to come along for the ride?

More than one person has pointed out that there are a lot of ulterior motives potentially at play.

Musk purchased Twitter with tons of debt, immediately removed a ton of safeguards against basically any content, and then was shocked that advertisers left the platform in droves. Last I heard, revenue was down 40% from pre-purchase. But it is worth noting: it’s not Musk that has the debt. It’s Twitter. In a leveraged buyout, the debt gets put on the acquired company’s books. So Twitter still has to pay interest on the debt. By merging Twitter with xAI in 2022, all of that debt got transferred to the AI company. And now, by merging with SpaceX, all the debt gets transferred to the rocket company.

This is a crazy amount of self dealing, and I think it’s fair to say that no one else in the world would be able to get away with it. In any sane world, Twitter would have died a financial death. The merger with Twitter was obviously bad for xAI’s shareholders, and is bad for SpaceX’s shareholders. Most of the time, that alone would be enough to sink a deal.

But the shareholders need to sue or otherwise claim damages. And the reality is, a lot of the shareholders of xAI and SpaceX are the same people who owned Twitter anyway! It’s all the same people!

Matt Levine has previously written about the Blackrock theory of governance. The basic idea is that because Blackrock (and equivalents) own basically every company in the world, they care more about the success of their overall portfolio than the success of any particular business. So, for e.g., during COVID, these massive institutional shareholders all demanded that Pfizer make the COVID vaccines available for cheap. Why? Because even though Pfizer individually stood to make tons and tons of money by withholding the vaccines, the broader Blackrock portfolio does worse when the world economy is shut down.

Almost 30% of Pfizer Inc.’s stock is held by Vanguard Group, BlackRock Inc., State Street Corp., Capital Group Cos. and Wellington Management Group. All of those are giant institutional investment firms that own shares of hundreds or thousands of companies, and those are just Pfizer’s biggest holders; lots of investors lower down the list are also huge diversified institutions. If Pfizer finds a coronavirus vaccine and distributes it as widely as possible—even at cost, even below cost, even for free, even at an enormous loss—it will make its owners richer by many many billions of dollars. BlackRock, for instance, owns about $16 billion of Pfizer stock. If Pfizer went to zero—if it bankrupted itself, selflessly producing and distributing vaccines—BlackRock (really its clients) would lose $16 billion. BlackRock owns about $2.9 trillion of other stocks; if a coronavirus vaccine allowed businesses to reopen and normal economic life to resume, and as a result those other stocks went up by 1 percent, that would more than make up for bankrupting Pfizer.

For BlackRock, I mean. BlackRock would be happy with that tradeoff, as would its clients, as would Vanguard and State Street and, in all likelihood, a majority of Pfizer’s shareholders, many of whom are diversified investors who own a lot of companies that aren’t Pfizer and are struggling.

I think the same general thing is true here. All of the shareholders in one Musk-verse company also own all of the other Musk-verse companies. So, broadly, they care more about whether the overall Musk-verse is doing well, rather than any particular Musk stock. Buying Twitter with tons of debt is a terrible idea if you’re a random investor. But buying Twitter with tons of debt and then hedging by buying SpaceX is a great idea! Musk used Twitter to get close with the Trump admin, and then got the Trump admin to pump billions of dollars into SpaceX through government contracts. Probably all of the Twitter investors were fine with this trade, because they all made more money on SpaceX than they lost on Twitter.

It’s a good question though. Coding agents are capable of miracles, so why has adoption been so slow, and why are so many senior engineers resistant to it? This question has consumed most of my waking hours for the last three months. Below, some meditations.

I.

A common argument is that the coding agents themselves aren’t actually good. They are not actually capable of miracles. The people who have “figured it out” are lying or are posers who aren’t doing real engineering. A slightly less aggressive but more common posture is to admit that coding agents are good sometimes, maybe for greenfield or prototype projects, but they are still a form of cutting corners that will eventually come back to bite.

Both of these takes are wrong. Everyone on my team is regularly running 5+ coding agents at a time. We are moving at a ridiculous rate. And yet our codebase has more test coverage and more integration tests than any other codebase I’ve worked with, including when I was at Google (what, you think my GNN training scripts had tests?). I feel very comfortable where we sit on the velocity/safety curve, and I’d even make the case that we are more safe per line of code or feature written than we were previously.

Take as given that the agents can do what it is claimed they can do. What then?

Another reasonable argument is that the agents are too hard to set up. The out of the box experience of using a coding agent is bad. The people who have coding agents figured out have invested a lot of time in tooling and learning. They figured out how to use git worktrees and figured out how to use web sessions and figured out how to write an AGENTS.md and figured out how to have an IV that is mainlining Twitter directly into their veins.

There is truth to this. Early coding agents were unwieldy and hard to control. Even though the models have gotten better, I still would hesitate to use these tools without any configuration. My team identified pretty early on that configuration was a bottleneck. It’s a pain. You have to learn an entirely new language, a new way of describing things, just to get your agent to follow instructions. Every time your agent doesn’t follow the right steps, you add more exclamation points to your config in the hopes that this time it will make a difference. We eventually figured it out, took us months. We built noriskillsets.dev and the nori CLI as ways to reduce friction, so other people don’t have to spend months. They work, try them out.

But still, difficult tooling in an immature ecosystem is not the whole truth. Nori tools are often necessary but insufficient.

After working with dozens of teams, size 1-100, I have two hypotheses about what is slowing adoption.

First: coding agents feel bad to use.

Second: when you use them effectively, the rest of the organization breaks.

II.

Why do coding agents feel bad to use?

My usual workflow involves running multiple agents at once. As many as I can fit into my computer RAM and my attention. Generally that’s about 5. I’ve been doing this for about 5 months now, so I’m very used to it.

Recently, I had to go back to using a single agent for a day, to experiment with how first time adopters experience the technology. And you know what? It fucking sucked!

When you have only a single coding agent, your natural inclination is to babysit it. That means you are sitting there, only partially engaged, watching text stream by. Maybe eventually you disconnect and start scrolling Twitter or something, and then you look up and half the day has gone by, you haven’t done anything, and you forgot what you were working on in the first place. The experience of using a single agent is one of being yanked around, like a kid on a rickety carnival ride. Moments of intense focus that punctuate long periods of boredom and disengagement.

I wrote about this before, in my post on Cathedral Builders:

In some sense, the point of a coding agent is to distance the engineer from the code they are writing. In some cases this is quite extreme — when I use a nori-powered agent, the coding agent is often chugging away autonomously for ten, twenty, thirty minutes before I check in on it. That’s great for me, because I’m not working on one extremely nasty extremely difficult problem at a time. I’m working on like 5 mostly straightforward problems. But if you’re the kind of person who’s primary workflow involves thinking super deeply about one thing, then even the simplest coding agent is designed to pull you out of that flow. A simple ‘hello world’ request from Claude Code takes ~5 seconds. Asking it to write ‘hello world’ to a python file takes ~20 seconds. In that time, you’re just sitting there twiddling your thumbs!

The coding agent is built to disconnect you from whatever you’re thinking about, which makes it uniquely bad for cathedral builders who need to think deeply about problems for long periods of time before they can make progress.

I strongly believe it is impossible to enter a flow state when managing a single coding agent. So the net result is that you are significantly less productive.

“Don’t babysit the agent!” says the true believer. “If watching every single action an agent takes is detrimental to productivity, let the agent go and do its own thing!” But now you’re right back to having to spend hours and hours figuring out how to set up a CLAUDE.md and SKILLS and a dozen other things. If you don’t do that — if you try to parallelize out of the box — you end up with unmaintainable slop.

I’ve been an eng manager for a few years now. In many ways, all of this is similar to how managers treat their team already. Junior engineers need guidance and handholding. Senior engineers get rubber stamp reviews on their PRs. I don’t mind when my TL wants to do a huge refactor (from a code perspective; I care a lot from the business angle). I’ve learned to let go of the code, make it someone else’s responsibility. That frees me up to do other things with my time. But that in turn can only happen because I trust my TLs.

I can only parallelize when I let go. I can only let go when I trust my agent not to screw everything up. And I can only trust my agent not to screw everything up when I’ve spent a ton of time configuring it. No wonder adoption is slow.

III.

So you’ve spent a while configuring things to exactly your specification. You trust the agent. You’ve learned to let go. You’re running 2, 5, 10, 20 agents at once. Things feel good. Then more of your team starts adopting this stuff, and instead of moving really fast suddenly things grind to a halt. What gives?

All of our best practices are designed for a world where code is expensive. First, the machines themselves were expensive. A server would take up an entire room and you’d have to carefully apportion compute time because running the machine was the budget of a small nation. Then the machine operators were expensive. Software became the critical industry and demand for software engineers shot up.

What happens in a world where code is expensive?

• We try to avoid working on the same files as teammates because we hate merge conflicts because figuring them out is hard because code is expensive.

• We spend lots of time in alignment meetings so people don’t accidentally work on the same things because we don’t want to have to close PRs because code is expensive.

• We do code review because we need to make sure more than one person on a team has a mental model of the code because the mental model is necessary to make any changes because code is expensive.

• We build extremely lean MVPs, stripping out anything that can be stripped out, because we don’t want to build in the wrong direction because code is expensive.

People are rewarded for doing these things and doing them well. People build their careers around these intuitions and are paid handsomely for bringing them to their teams. People identify with these things. “I’m good at code review” is absolutely something people are proud of!

Code is no longer expensive. Code is cheap. Our intuitions are operating in the wrong evolutionary environment. So if you bring engineers who are aggressively using coding agents into a process built around mitigating the cost of code, you’re going to hit a wall. The engineers will literally just idle as they get bottlenecked by process. Things that are ostensibly meant to speed things up end up getting in the way.

IV.

So the dev process needs to change. In some places, radically. We’re still figuring out what exactly this means at Nori, but some principles that have helped:

• Don’t wait for others. If you need to get something done that depends on someone else, just duplicate the work and finish the feature. Worst case scenario, you have to close one of two PRs. Since code is cheap, whatever; stepping on toes doesn’t matter.

• Code review shouldn’t be required. Accept that the mental model for everyone is going to be a bit more hazy. It’s fine, because it’s much easier to just ask an AI what any given piece of code is doing.

• Focus on product over implementation. Confusion about what is being built is the most common source of slowdown. Spend time making sure everyone understands the user stories that define your product. All of your engineering meetings should be laser focused on product.

• Let people build things. An MVP no longer has to be minimal. If you do the user stories thing right, you’ll get way more mileage out of letting your team go wild.

In general, we try to aggressively remove bottlenecks to parallelization. Increasingly, those bottlenecks are not on the dev side.

As a concrete example, I said that understanding user stories is critical for moving fast. User stories come from the product folks, not the eng folks. So much of product direction is defined by slow ab tests and consumer validation studies that can only be justified when engineering epics take weeks or months. When your engineering team can ship daily, your product cycles simply need to be faster. Way faster. So far, we’ve found success by using AI to dramatically increase and automate the scale of outreach and feedback collection in our product motion. But that just bumps the bottleneck to somewhere else. When the product team can cycle quickly, the sales team ends up being the bottleneck. And so on.

This process is exactly like debugging a slow database request or api call. Find the slowest part of the waterfall diagram, aggressively optimize it, and then repeat. I think every part of every organization is going to radically change in the next year; how can it not? The pressure to move faster will come naturally as individual teams start adopting AI tooling and then bump against the teams that haven’t yet. That in turn means that best practices across the board need to be rethought or thrown out. Exciting, because it’s a lot of opportunity to discover what the new best practices are. Scary, because upheaval is always scary.

V.

I opened by positing that teams are struggling to adopt AI. I think that’s true, but it’s also kinda a lie. People are slow to adopt AI compared to what? If you zoom out a bit, adoption of AI workflows is happening at an insane clip. Claude Code only hit general release in May of last year — only 8 months. Maybe ChatGPT had a more meteoric rise? I can’t think of other products that have this steep an adoption curve, certainly not dev tooling.

But everyone feels behind anyway.

There is a massive amount of FOMO in the market right now. Everyone keeps hearing about these crazy gains to productivity. They can’t keep up with Twitter. And even if they could, they wouldn’t be able to identify what works and what’s garbage (the vast majority of it is garbage). I think eventually all that chaos will stabilize as best practices harden. But that doesn’t mean companies are safe to wait it out — just the opposite. Small, nimble teams that remodel their business around AI are gunning for big established players. In this rare instance, I think the FOMO is well placed. So if you’re one of those executives or leaders asking “why won’t my team use agents?” this post will hopefully be a wakeup call. Being unable to adapt to a world with AI is now your company’s biggest hair on fire existential problem.

PS: I recently went back and reread my old ‘meditations on AI’ post. I stand by basically everything I wrote back then, and think understanding that post is also highly relevant to what’s happening now. But instead of quoting the whole thing, I’ll just link it here.

Why does Anthropic struggle so much with rendering monospace text?

Why does Anthropic — the frontier AI lab valued at billions of dollars, owner of the most sophisticated AI coding tools, and the undisputed king of agents for 2025 — struggle so much with rendering monospace text?

From paper teletypewriters to modern digital emulators, terminal interfaces have always printed out a simple monospace text grid. This avoids some of the toughest problems in text rendering: the styling is heavily constrained, you often have a single font, one or two colors per grid cell, and the layout and shaping is relatively trivial.1 In the words of Casey Muratori:

Drawing a monospace terminal display is straightforward....

[8 sentence pseudocode for the full rendering pipeline...]

...That’s it, right? I mean that is the entire renderer.

Given these expectations for terminals, there was consternation from devs when details surfaced on X about the Claude Code rendering architecture:

Most people’s mental model of Claude Code is that “it’s just a TUI” but it should really be closer to “a small game engine”.

For each frame our pipeline constructs a scene graph with React then
-> layouts elements
-> rasterizes them to a 2d screen
-> diffs that against the previous screen
-> finally uses the diff to generate ANSI sequences to draw

We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written.

Responses ranged widely in tone, but shared a similar message.

...It’s one of the worst technical things I’ve ever heard, posted with a straight face...

Sometimes, developers get hyper-focused on tackling the immediate technical challenges in front of them that it prevents them from reflecting on past decisions.

It’s like you take a fork in a road, and find you need to climb a mountain.

The faster route to your destination may be to backtrack, and take a more direct route without a mountain in the way...

We apparently live in the clown universe, where a simple TUI is driven by React and takes 11ms to lay out a few boxes and monospaced text...

During the storm of criticism, there was at least a little bit of levity:

Layout 0.9ms

Rasterization 2.1ms

Diffing 1.2ms

Building the React scene graph 11.0ms

Generating ANSI sequences 0.8ms

someone who is good at the computer please help me budget this. my small game engine is dying

But many of the responses were even ruder (because some people really get off on being smug and critical on the internet). I think there’s slightly more to it than that, and the Claude Code team obviously had reasons to go in the direction they did. Instead of coming in with another dunk (React? hahaha amirite), let’s see what there is to learn if we think about the tradeoffs and limitations that they’re working with.

Before generalizing to some broader comparison points like React vs Ratatui, garbage collected vs not, interpreted vs compiled, I’ll begin with some concrete estimates for this type of workload, and why the rebuild around diff-based rendering makes sense.

Speed of light (text) in the terminal

Any terminal interface is fundamentally limited by two things. To understand this, it’s important to consider the basic pseudoterminal (pty) architecture. A program called a “virtual terminal emulator” like iTerm2, Ghostty, VSCode integrated terminal, etc, is used to run or view any terminal process like Claude Code, NetHack, cat /dv/urandom, etc. To connect these two, the OS opens up a bidirectional communication channel with a “master” end (the terminal app, what I’ll call the emulator) and the “slave” end (the foreground process group, what I’ll call the process)2 Data sent from the emulator to the process appears as stdin. Conversely, the process writes to stdout and stderr, which the emulator receives and parses to update the display. All of these are byte streams containing a mix of Unicode, ANSI escape sequences, and newer and rarer terminal protocol escape sequences.

So imagine you’ve built a program that will render a terminal interface. Based on the pty architecture, the two limits on the rendering speed are:

1. How fast can your program process and respond to new data? Including stdin data like key presses and window resizes, or other data like reading files, network responses, and subprocess results.

2. How fast can everything that is *not* your program, mainly the emulator on the other end, receive and render your output bytestream? Technically the pty communication itself also falls into the “not your program” category and also includes some work (like the line discipline or context switches), but this is nowhere close to the emulator’s own bottleneck.

Unrelated, but did you know there’s an actual hard limit on how many terminal interfaces you can have open?! From the Linux pty manpage:
```
$ man pty
...
The Linux kernel imposes a limit on the number of available UNIX
98 pseudoterminals. Up to and including Linux 2.6.3, this limit
is configured at kernel compilation time (CONFIG_UNIX98_PTYS),
and the permitted number of pseudoterminals can be up to 2048,
with a default setting of 256. Since Linux 2.6.4, the limit is
dynamically adjustable via /proc/sys/kernel/pty/max, and a
corresponding file, /proc/sys/kernel/pty/nr, indicates how many
pseudoterminals are currently in use.
...
```

Time for some napkin math. Imagine we want to rerender as many lines of history as possible: what limits should we expect the bottleneck in the emulator to impose on us? Looking at one of my chats, a 150,000 token chat history contains about 53KB of printed text. The full count of tokens would be more like 1MB of text, but the majority of that context contains the thinking tokens (not shown) and the tool calls (truncated), so the output is much smaller. However, not shown in the raw 53KB of text are the escape codes for colors, bold or dim, and ANSI codes for movement and clearing and printing; altogether this could conceivably double or triple the size in raw bytes.3 So we’re looking at something in the low hundreds of KB across the pty connection, for the size of each frame of full history render. Assuming a 60 FPS experience, we’re looking at ~10MB per second (for example, when an active and updating history cell is just off screen, the frequent cause of Claude Code flicker last year).

So how close is this to the theoretical max throughput of the emulator? Using the cmuratori/termbench repo, I can compare a few terminals’ max throughput on colored text (see Appendix: Emulator Throughput). To summarize the results from below, modern emulators on modern hardware show result between 30-100MB/s, and legacy emulators are at least an order of magnitude less than that. The GPU-accelerated terminal emulators on my laptop are right around the median of ~50-60MB/s. Non-GPU-accelerated terminals (that instead allow synchronous frame drops) show similar throughput. The similarity is unsurprising; the bottleneck doesn’t come from the rasterization but from the singly threaded CPU-bound logic to track the state machine of ANSI codes. The variance in my results, at least for the Linux ecosystem, correlates far more with hardware than with the emulator. The built-in terminal versus GPU accelerated terminal on my Mac Mini shows another performance gap: 30MB/s versus 65MB/s. I don’t have the appetite to benchmark Windows today, but trust me when I say it’s generally a cut below the Unix systems. Finally, on a raw Linux kernel TTY (e.g. ttyN) without graphical emulation, using minimal device drivers, no GPU acceleration, and no logic for synchronous frame drops, we see the legacy graphics bottleneck: just over 100KB/s. Not per frame, per *second*.

So the napkin math puts us in 10-50MB/s territory for most setups, and the dreaded flicker scenario (an updating cell offscreen, and a naive history reprint) requires up to 10MB/s. This doesn’t fully preclude the possibility of a full history rerender for newer setups, but it’s not looking good! These benchmarks show a theoretical maximum that is within the same order of magnitude as the required throughput. This evidence does guarantee the full history rerender will be painful on some legacy terminal emulators or very old hardware, and it will likely introduce frame drops and brief flickers on all but the most optimized setups. The important detail here is that these limitations are completely absent any implementation choices of the code itself. This is the maximum speed of the pty architecture, for the requirements the Claude Code team is operating under.

This was the most ignorant part of the critique last week, that lampooned the new architecture, or the software choices. I question the requirements the Claude Code team has set, but to implement those requirements they absolutely need a new architecture like they described. Game engine programming is often impressive because after saturating performance bottlenecks in the hardware or OS or another abstraction outside your control, you need to design a smarter approach for your workload. In that light, Thariq’s comparison is fair.

The new architecture

A retained mode renderer

The details of their new architecture solve this throughput issue. Rather than forcing entire documents down the pty bottleneck, the new architecture implements the terminal equivalent of a retained mode interface. The emulator retains the screen state between frames, and the program is expected to push down only the minimal ANSI codes to move the cursor (CUP) and update specific cells. Anyone familiar with the Ratatui library knows this pattern. The application still describes the UI every frame, but the new renderer only transmits the changes.

As compared to the React virtual DOM:

- On the Web: DOM updates are slow, so we use VDOM to minimize browser repaints.

- In the Terminal: The byte stream is fast, but the parsing is the bottleneck.

By treating the terminal as a grid of cells rather than a tree of objects, the terminal avoids any “parser jitter.” However, implementing this inside a React-like framework means they are using a nested object tree. Not only is this algorithmically more distant from the text grid, it is still creating thousands (tens of thousands?) of short-lived objects per frame to represent that grid of characters. The new design has fixed the I/O bottleneck, but they likely increased the GC pressure, moving the accidental complexity elsewhere.

This leaves us with the remaining, more permanent issue that Claude Code is now dealing with: fighting the framework.

Let history be history

Terminal emulators are optimized for two distinct scenarios.

1. The Scrollback: the program turns information over to the emulator to own. Immutable, fast, and forgotten by the program. The pty is explicitly optimized for direct copyover of plain Unicode data.

2. The Viewport: information fully owned by the app. Interactive, temporary, easily customized UI, dynamic rewraps. The altscreen mode offers a separate buffer, fully controlled by the foreground process (like neovim or Emacs or top). Limited height viewports can also be implemented below the history (like `fzf --inline`).

Claude Code has decided to violate this contract. It decides to treat the *entire* scrollback as a reactive viewport. Ostensibly, there are some benefits over altscreen: native search (cmd-F or similar), native text selection, native clipboard interactions, or a longer list of multiplexer features. But the cost is fragility.

When the context gets too long, Claude “compacts” the history. In doing so, it often clears the *entire* terminal scrollback, including history from before the Claude Code session! Similarly, “fake fullscreen” features (like ctrl+o extended output) rewrite the history instead of using altscreen, so any failure scenario risks a broken scrollback that you can’t reference or copy from. The most convenient feature is history rewrapping on window resize, and that is gone the moment you quit Claude Code, leaving you with the static history artifact that you were briefly told not to settle for!

Non-pessimization of the requirements

Pessimization involves crippling the performance of your program, either by assuming requirements that don’t exist, or ignoring constraints that do. There’s an attempt here to support an ambitious set of requirements, and I applaud the Claude Code team for pushing boundaries on terminal experiences. However, every one of the tradeoffs I just described around fragility and fighting the emulator is just not my preference as a user. If I want a scrollable dynamic interface, there is altscreen. If I want a dynamic viewport below longer history, there are normal stdout conventions. If I need the rewrapped history, I can /resume the conversation in a new window. Heck, even introducing a separate /rewrap command would make me happier than a system that constantly fights the terminal emulator for custody of the pixels.

In their design, they’ve pessimized the wrong thing. Their initial requirements assumed that rerendering is free, and responsiveness are paramount. This is incorrect — the history rerenders have proved to not be free, and I only care about responsiveness in the small section at the bottom that I interact with!

I’ve found that OpenAI struck a better balance with the Codex CLI design, using a double-buffered history for active vs completed cells. It brings Ratatui performance with diff-based rendering for free, has zero-overhead thread safety, and the raw speed of a non-GC compiled language.

Full disclosure (full plug at the end), I’ve been working on a distant fork of the Codex architecture to wrap the Claude agent and other agent choices. It feels nice using Opus 4.5 without flicker, stuttering, artifacts, or my terminal history getting cleared!

Some takeaways

Based on these tidbits from the Claude Code team, I imagine they now have the full diff-based TUI rendering engine built out in TypeScript. I don’t know in which areas it’s at feature parity, superiority, or inferiority with similar frameworks like Ratatui. But having seen the general shape and complexity of code that goes into Ratatui, it is a great milestone for the Claude Code team to deliver their own diff-based engine. Despite the immature reactions online, this was definitely a net positive for Claude Code, and puts them much closer to my performance expectations for a cutting-edge TUI. While I consider the requirements they’ve inflicted on themselves to be not my personal cup of tea, I appreciate them experimenting with the limitations of interactivity in the terminal.

There are a few takeaways I’m left with, if I had to offer my remaining 2c (on top of the 98c above):

1. Why fight the framework: Every feature that modifies history risks destroying it. Terminal users have learned to trust their scrollback as immutable log, and I personally don’t like when programs break that trust.

2. Accidental complexity moves elsewhere: I still question if rendering a text grid via an object tree (DOM/React) is appropriate for an ANSI terminal. With a long chat history, creating or destroying thousands of node objects can easily cause a GC pause, and a frame drop may exhibit a blank screen or partially drawn UI — more flicker!

3. Language tradeoffs: Interpreted languages often justify the performance hit with “portability.” But in the terminal, the “non-standard environments” (Windows, old Unix distros, Android devices, etc) are *exactly* where you need the fastest performance... because they have the slowest emulators!

Despite the improvements to their architecture, I can’t say I would lean on a GC interpreted language like JS for this type of work.4 Clearly they are thinking about some of these concerns, given the acquisition of Bun. And clearly they are undeterred by the risks, given... The acquisition of Bun. I’m impressed by the work within the Bun project to date, and it will be interesting to see what their next chapter within Anthropic looks like. As it relates to Claude Code, this feels kind of like buying more RAM because your program has a memory leak. However, I have to imagine it’s a strategic decision that goes beyond just one product team, and they see a broader need in their portfolio for performant JavaScript runtimes.

P.S., the Nori Approach

If you’re interested in these topics, or just want a terminal agent that doesn’t flicker, check out the work we’re doing at Tilework!

Our Nori CLI is a distant fork of the Codex CLI, with support for Claude Code. We built this on the philosophy that tools should be fast, intuitive, and respect your terminal’s native behavior. We’re working on building the things that make us faster at building the things that make us faster, and tools like this are part of our long term vision to help developers fully own (and enjoy) their own toolbox.

Appendix: Emulator Throughput

Thanks to Casey Muratori’s termbench project!

These throughputs are explicitly *not* the output rendered per second; they are the input “accepted” per second. Because of this, they provide a guaranteed max on what your terminal could render out, but do *not* necessarily establish a viable minimum. For example, COSMIC achieves the top result, but actually exhibits long stutters and freezes throughout the benchmark.

• ManyLine: Throughput when printing many short lines.

• LongLine: Throughput when printing long lines wrapping the screen.

• FGPerChar: Throughput when changing foreground color every character.

• FGBGPerChar: Throughput when changing foreground and background color every character.

• TermMarkV2: A balanced benchmark simulating complex terminal usage.

All numbers in megabytes per second. The laptop environment is running Pop!OS 24.04 with a Ryzen 7 6800U. The desktop environment is running Pop!OS 24.04 with a Ryzen 5 7600X. The Mac Mini environment is running macOS 26.2 with an Apple M4 CPU.

Notes:

• Virtual tty3: This benchmark ran on a virtual TTY (`/dev/tty3`) without a windowing system. It used the `TermMarkV2 Tiny` profile (in order to complete in under 10 minutes), whereas all other emulators used `TermMarkV2 Normal`.

• Highest Throughput: The highest value for each column within a specific environment is **bolded**.

What is the core of finance? To buy low and sell high. When you buy a stock, or an option, or an oil field, or ten trucks of corn, you are making a prediction about the future. And then you get rewarded if you are correct. Our financial system incentivizes people to come up with better and better forecasts of the future. The folks over at Jane Street pay hundreds of thousands of dollars for the best meteorologists in the world to analyze cloud data in Tibet so they can predict what will happen to Indian and Chinese commodities weeks and months later, while the quants at Citadel are ingesting satellite data to see which factories have trucks rolling out and which ones don’t so they can predict supply chain issues a year out.

This information ends up disseminating through the market. When Jane Street sells all of its Indian commodities, they are sharing their prediction about the future, which in turn is reflected in the price. So anyone in the world can eyeball the price movements of an asset at any time, and from that predict all sorts of interesting things about what may happen and why. Collective wisdom emerges from the populace’s shared interest in making money..

It’s not perfect. The markets can be wrong. But there is some truth in the semi-viral WallStreetBets post Everything is priced in.

Don’t even ask the question. The answer is yes, it’s priced in. Think Amazon will beat the next earnings? That’s already been priced in. You work at the drive thru for Mickey D’s and found out that the burgers are made of human meat? Priced in. You think insiders don’t already know that? The market is an all powerful, all encompassing being that knows the very inner workings of your subconscious before you were even born. Your very existence was priced in decades ago when the market was valuing Standard Oil’s expected future earnings based on population growth that would lead to your birth, what age you would get a car, how many times you would drive your car every week, how many times you take the bus/train, etc. Anything you can think of has already been priced in, even the things you aren’t thinking of.

(Note: not financial advice)

Though bankers are a favorite punching bag, I think the folks working in finance are doing an important job. It is really valuable to be able to predict the future. I think it makes the world a more stable place. People want to make prices go up, so they want to do things that create value and reduce things that destroy value. Even the most authoritarian dictator has an eye on the public markets because of their and their cronies’ investments; if the markets tank because of something they do, they will likely chicken out.

Except, wait, that’s not right. Sometimes people don’t want to make prices go up.

In an unregulated market, executives and politicians and people in power have pretty strong financial incentives to tear everything down. The better a company is doing, the more money to be made shorting the stock. The safer a country looks, the more money to be made destabilizing the place with war. The financial ecosystem as a whole is composed of billions of individuals, each with their own wants and desires. The markets are a prediction machine that drives to averages. You make a ton of cash if a shit company gets better; but you also make a ton of cash if a great company gets worse. The purely financial incentives stabilize somewhere in the middle.

So we try to leash the markets. We carve out certain kinds of trades as illegal. Fraudulent trades, misrepresentation, trading on insider information. If you’re a CEO and you buy a ton of shares in your competitor’s company days before you announce that you are going to set fire to your factories, you go to jail. We create ‘fiduciary duties’. Business executives have obligations to their company that are legally enforceable. As do lawmakers and members of the judiciary and so on. We do all this to ensure that our future prediction machines stay finely tuned, and we mostly succeed. You need the leash, even if it makes the markets less efficient, because the alternative is to encourage chaos.

I am against prediction markets, because prediction markets encourage chaos.

Prediction markets are a new kind of betting platform where individuals can bet directly on what is going to happen in the future. Generally, someone will pose a question, like “Will Dems take back the house in the 2026 midterms?” And the ‘crowd’ will trade shares marked “yes” and “no”. At some point in the future, the question will resolve. We will know, in November of 2026, whether the Dems have taken back the house. At that point, folks who have shares of the correct answer get a payout.

So people try to predict the future. In August of 2026, if it seems like the Dems are doing great, ‘yes’ shares will trade higher because people want the payout. And if it seems like the Dems are having a bad time, the ‘no’ shares will trade higher.

If this sounds like sports betting, that’s because it is basically the same beast. There’s a pretty smooth gradient between “I am going to put money on the 49ers because I think they will beat the Eagles” and “I am going to put money on Google because I think they will beat OpenAI”. Of course, there’s a lot of regulation around putting money on Google. And in recent years, there is a lot less regulation around putting money on just about anything else.2

This has led to a set of “prediction market companies”, including Kalshi, Manifold, Metaculus, and Polymarket.

Proponents argue that these markets will benefit people the same way financial markets already do. The steelman case is something like this:

• Having accurate pricing is a good thing because it helps people prepare for the future;

• When insiders share their information with the broader world, it results in more accurate pricing;

• Therefore, incentivizing people with private information to bet on prediction markets makes everyone better off.

It’s a tempting argument, especially since people3 are already looking for sophisticated-sounding excuses to bet on things. The problem is, the steelman completely dodges all of the very real problems that come with incentivizing betting.

Here is an example. A few weeks ago, the Trump admin kidnapped Venezuelan dictator Nicolas Maduro. We can do a dozen rounds about whether or not this was a good thing, it’s not really relevant. What is relevant is that moments before Maduro’s capture was announced, someone went and bought a ton of shares on Polymarket saying that Maduro would be out of office before end of Jan.

The person in question made over $400k. Not the most they could have made if they were really looking to trade serious money. But a tidy sum to pick up entirely risk free.

This raises some questions. We don’t know who the guy was, nor do we know what role they played in the decision to capture Maduro. Most likely, it was some random grunt who heard the news by being in the right place at the right time. But there’s a specter of doubt — what if it was a member of the Chiefs of Staff? A member of the Cabinet? The President himself? We generally seek transparency in the wheelings and dealings of our political class precisely so that we can track skewed incentives. Prediction markets can skew everyone’s incentives, the same way as straightforward manipulation of financial incentives.

Free markets drive towards averages. Bad companies become better, but great companies become worse. What does that mean for stability? Despite the news, we live in one of the most stable geopolitical eras. Average life expectancy keeps going up, average wealth keeps going up, and poverty and child mortality and so on keep going down. Prediction markets add financial pressure to reverse those trends. Tomorrow, someone could put up a prediction market on the likelihood of a wolf attack in a major French city. This is an extremely unlikely event, so most people will bet against it. Which, of course, means there is now a massive financial incentive for someone to go and release a bunch of wild wolves on the streets of Paris. Or, put a slightly different way, there is never going to be another American invasion or police action that doesn’t make some Pentagon guy rich enough to buy a boat.4

Take a look at some of the markets on these sites. US strikes in Iran:

Cloudflare critical incident:

Bets on earnings:

No one cares about, like, a betting market on whether Bad Bunny wears a skirt to the superbowl.5 But it’s scary to imagine someone with decision-making power having a personal investment in the market outcomes above. Maybe it’s just because I am in tech, but the cloudflare one seems particularly egregious. It just isn’t that hard for a single engineer to push malicious code! And if they do in the next week, they can make a guaranteed 28x return.

Insider trading laws are already tough to enforce. But historically, the number of decision makers who have a lot at stake is mostly pretty small. Prediction markets feel different in two critical ways. First, a lot more people can have a direct impact on a prediction market. Second, there are a lot more decisions that can now be directly traded on. This one-two punch means the surface area to cover is way higher. Before, it was pretty hard to ‘trade’ on your ability to create an outage on Cloudflare, even if a lot of people could or would do so. Now, anyone can make money off destabilizing behavior. It’s grim to apply the same thought process to, e.g., mass shootings.

From where I sit, prediction markets need a massive amount of regulation before I begin to feel good about them. But there are obvious low hanging things that we can do, like:

• Deanonymize prediction markets;

• Pass legislation or guidance indicating that prediction markets behave like securities;

• Pass new legislation emphasizing that insider trading is still illegal

It’s weird to focus on this in the midst of everything else going on, but it all feels connected. There is no better symbol of the gamification and hyper-optimization of our lives than a platform where you are encouraged to bet your life savings on how many times Elon Musk will tweet in a given day. Or whether the ICE shooter in Minneapolis will be charged by the end of the quarter.

Our company ethos is “build the things that make us faster at building the things that make us faster.” This is a necessary part of any startup, of course. But we take this to extremes. When LLMs failed to adapt to larger codebases, we built the first versions of nori-skillsets to get to the point where agents were writing all of the code at our company. When Claude Code failed to build out critical features like built in notifications and SessionEnd output, we built on top of it to let us run >10 agents at once. And when our agent CLIs began to lag behind, we built our own CLI tool in rust so that we could keep pace with our own throughput. We don’t wait for the industry to catch up. We don’t have time for that. AI is moving too fast for that.

A few days ago, I complained that the industry and the ecosystem needs a trusted package manager

Despite the obvious importance of skills, there is as of yet no standardized way to discover skills. For example, if you search for ‘creating-skills/SKILL.md’, you will find 76 different versions of a skill for creating skills. To try and find good skills, people have taken to producing massive lists of dozens of otherwise-unrelated skills on github. It’s giving very Web 0.1. The problem, of course, is that you also can’t find those lists! There are 53 different repositories called ‘awesome claude skills’!

Most of these skills are bad. A huge huge chunk of them are written by AI without any guidance. These are possibly the worst, because AI tools do not yet know how to write good skills…

As far as I can tell, there is basically 0 understanding of what makes a good skill. So what proliferates in the market is slop, and search indices for that slop. It happens to me all the time. I will see a skill that I think is cool, try it out, and realize that it doesn’t work. Or that it actively makes my agent worse. And then I either scrap it, or spend the time rewriting the whole thing from scratch…There is a need for an AI npm or pypi, a package manager that is an ecosystem backbone, one that does a good job surfacing the best tools instead of simply indexing all of them without care for quality.

More succinctly, the ecosystem needs a place where developers can:

• Quickly and easily get the latest configuration options and tooling for their agents;

• Make it easy to onboard their teams to agentic coding best practices;

• Avoid the slop on GitHub;

• Not have to spend hours and hours becoming experts in these tools before being able to get value from them.

We heard this from our own customers over and over again. “My team won’t adopt coding agents because they had a terrible first impression and don’t have the time to learn the difference between a skill, a subagent, a slash command, and an agent.md.” I don’t blame those engineers! If I wasn’t already in AI, I wouldn’t touch this ecosystem with a ten foot pole.

Well, we build the things that make us faster. And a package manager that we can use to share configs within our team definitely makes us faster. We built an initial version of this thing a month ago and have been using it in house ever since.

After my post on Sunday, we realized no one was coming to save the wider ecosystem. So we’ll do it. We’re bringing the things that make us faster to the public so everyone can use them.

I’m very excited to announce noriskillsets.dev.

The goal of noriskillsets.dev is to provide a curated and off-the-shelf package manager for the AI ecosystem. You can:

• Create skillsets (collections of skills)1 that have the tools and configs you need

• Easily switch between skillsets

• Source the best versions of skills and other tools, as vetted by the Nori team

• Get internal/private registries to easily version control and share configs with your team.

Some people like to brag that they’ve indexed thousands of skills. Not us. Right now, we have ~33.

If you’re just getting started with agentic coding, noriskillsets.dev will help you become a master. If you’re trying to get your team to use coding agents, noriskillsets.dev will make your settings portable and save your engineers hours of configuration time. And if you’re trying to stay abreast of the fire hose, don’t worry — noriskillsets.dev will keep you up to date on the latest and greatest.

The ease and simplicity of NPM with the quality guarantees of the Apple App Store.

Try it out, let us know what you think. And if you have skills or configs that you want to share, or if there’s something out there that you know works that we’re not aware of, send it our way.

PS. Eng team of 3, 28 PRs landed in the last 24 hours. Our skillsets have us moving so fast.

See also:

Hacker News Announcement

usenori.ai mainpage

What is a Skillset