OpenAI was trying to test the cybersecurity capabilities of its models, including one that hasn’t yet been released to the public. OpenAI asked the models to t…

Read more

During last year’s competition, Polish programmer Przemysław Dębiak (known as “Psyho”) narrowly claimed first place in the heuristic division. He beat 11 human competitors — and an internal OpenAI model trained to be especially strong at reasoning tasks.

“Humanity has prevailed (for now!)” he wrote in a tweet right after the competition. OpenAI’s model came in second after leading for most of the 10-hour competition, a surprisingly strong result for AI models at the time.

OpenAI’s models last year weren’t good enough to compete in the algorithmic division.

The 2026 competition, held this week, turned out very differently. Organizers chose a heuristic problem designed to help humans succeed. Despite that, OpenAI “completely demolished human competitors,” Psyho noted after the two-day competition finished Wednesday night. It’s hard to quantify exactly how big the AI’s margin of victory was, but Psyho told me that he would guess that humans would need to work at least a few more days to match the AI’s score — though he stressed that this is a hard number to predict exactly.

The next day, OpenAI’s system crushed humans on the algorithmic problems as well. Over the course of the seven-hour competition, it solved all five problems, including two that none of the 12 human competitors — all among the best in the world — were able to solve.

So at the award ceremony for the 2026 AtCoder competition, the organizers presented two “humanity surrenders” awards to OpenAI for its models’ performances in the two competitions.

This was probably the last time humans had a realistic shot at winning a programming competition against top AI models. Today’s AI models can find impressive, elegant solutions much more quickly than humans. And future models will only get better.

This performance was very impressive for OpenAI

In some ways, OpenAI’s performance was even more impressive than the raw score suggests.

Read more

For ethical reasons, I won’t accept ads for companies we are likely to cover. That means you won’t see ads for name-brand companies like Anthropic, Waymo, or Microsoft. Advertisers won’t be able to buy favorable coverage, and we’ll always make it clear what’s an ad and what is independent editorial content. I’ll post details more about this before we do our first ad-supported post.

I’ve never done this before, so I’m looking for experienced people who can give me advice — and perhaps help me sell the ads as well. If that’s you, please email me: tim@understandingai.org.

If you represent a company that would like to advertise on Understanding AI, please contact me at the same address.

Back in March, I surveyed readers to better understand our audience. At the time, we had around 190,000 readers, and more than 1,000 of you responded. Thanks to all who participated!

The main takeaway from the survey is that advertising on Understanding AI is a great way to reach influential and tech-savvy readers:

• 25% of respondents were engineers, scientists, researchers, IT professionals, or others doing technical work.

• Another 15% are founders, executives, or managers.

• 19% of respondents say they have control over technology budgets at their companies, while another 25% say they recommend or evaluate technology for their companies.

• Some respondents control or influence substantial budgets: 3% say they control or influence budgets larger than $5 million, while another 4% control or influence budgets between $1 million and $5 million.

Read on for detailed results from the March survey.

Subscribe now

1. A lot of readers found us via Substack

How do people find Understanding AI? Nearly half of respondents say they found us thanks to a recommendation from Substack itself. Other Substack-based newsletters — including Nate Silver, Derek Thompson, Sayash Kapoor and Arvind Naryanan, Noah Smith, Matt Yglesias, and Joey Politano — have each driven thousands of signups. An interview with Ben Thompson (who isn’t on Substack) drove hundreds of signups in 2024. My former employer, Ars Technica, accounts for about 4% of respondents.

Two social media sites — Twitter and LinkedIn — accounted for 15% of our respondents. Search engines, word of mouth, and a long tail of other sources round out the list.

2. Readers care about LLMs, technology deep dives, and AI infrastructure

What do readers want to read about? This chart shows the topics readers say are most likely to hold their attention. Unsurprisingly, LLMs top the list, with technical deep dives, industry analysis, and AI infrastructure close behind. Readers are also interested in “softer” topics such as AI policy and the impact of AI on the labor market.

At the opposite end of the spectrum, readers continue to have fairly low interest in self-driving cars, robotics, and the semiconductor industry. I’ll be honest — we’re not going to give too much weight to reader preferences here. Not only is self-driving an important industry in its own right, I believe studying it can provide insights into the problems facing frontier model developers today. And we hope our forthcoming series on robots will convince readers that robotics is an interesting topic.

3. Our readers are technically sophisticated

A wide range of people read Understanding AI, from students to retirees to doctors and lawyers. But I was particularly happy to see strong representation from engineers, entrepreneurs, and corporate executives. In the chart, I’ve colored engineers and scientists red, while manager and executive are blue.

These red and blue bars represent the folks actually building AI technology. I love having these folks as readers because these are the folks who will complain if we get the technical details wrong. I think they will be also be appealing to advertisers, since they often hold the purse strings of corporate IT spending.

4. At least 27% work in technology, research, or academia

We have readers from a diverse range of industries. Some work directly on AI, either as academic researchers or at companies building AI products. But we also have a lot of readers in other industries, including education, health care, and the investment world.

5. 27% are actively involved in AI-related research or development

A significant minority of respondents — 27% — say they are actively involved in developing and deploying AI systems.

6. Readers have a lot of influence over corporate IT spending

Our readers exercise a lot of influence over technology spending at their companies. Nearly 20% of respondents say they have final authority to approve technology purchases. Another 25% are involved in recommending or evaluating technology products.

7. Readers control budgets as high as $5 million

About 7% of respondents say they control or influence more than $1 million in spending each year — including 4% who say they control or influence more than $5 million in spending. Another 15% influence budgets between $50,000 and $1 million.

Conclusion

So if you represent a company interested in advertising on Understanding AI, please get in touch by emailing tim@understandingai.org. I’d also love to hear people who are interested in helping me sell ads.

Read more

The government, Anthropic said, had “issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States.” Because Anthropic doesn’t have a way to limit access to Americans, this amounted to a de facto ban on the technology.

Neither Anthropic nor the US government has provided much detail on the order’s rationale or legal basis. But over the weekend, a number of news organizations published articles describing the negotiations that preceded Friday’s announcement. The most detailed was this Saturday article in Politico that described a “frantic 24-hour effort by senior officials to convince the company to voluntarily pull a newly released artificial intelligence model that officials believed posed security risks.”

Multiple news outlets, including Politico and The Information, have reported that Amazon CEO Andy Jassy alerted the Trump Administration about potential vulnerabilities in Anthropic’s top models. Amazon apparently discovered it was possible to bypass Fable’s guardrails and thereby gain access to some of the powerful cybersecurity capabilities Anthropic has withheld from the market since the April announcement of Claude Mythos Preview.

Politico reports that during a Friday call, Anthropic CEO Dario Amodei “pushed back on the administration’s concerns, defended the guardrails, and argued that the type of bypass that occurred, which he believed to be specific, did not pose the same risk as a broader jailbreak.”

Anthropic made similar points in its Friday post announcing the suspension of Fable access: “No testers have yet been able to find a universal jailbreak — a jailbreak method that can very broadly bypass the model’s safeguards, unblocking a wide range of cyber capabilities.”

But according to Politico, senior administration officials were unmoved by Amodei’s arguments. They slapped export controls on Anthropic’s most powerful models.

This is the second time the Trump Administration has taken dramatic legal action against Anthropic. Back in February, the Defense Department declared Anthropic to be a supply chain risk, effectively prohibiting use of its models by the military — as well as certain military contractors. That action has been tied up in court ever since, with a federal judge wondering whether the government’s rationale was pretextual.

“Nothing in the governing statute supports the Orwellian notion that an American company may be branded a potential adversary and saboteur of the US for expressing disagreement with the government,” wrote Judge Rita Lin in a March ruling.

In a new episode of my podcast, AI Summer, the legal scholar Alan Rozenshtein told me that Friday’s export ban may be on firmer ground, legally speaking.

“What the government is doing from a legal perspective is facially plausible,” he said of Friday’s order. “They do really have these export controls, and these export controls really can create a de facto licensing regime.”

So the Trump Administration likely has the power to seriously harm Anthropic if it wants to do so. The big question is whether Trump wants to do that.

Read more

The announcement that got everyone so mad? Anthropic was planning to subtly degrade the quality of responses to prompts that appeared to be “targeting frontier LLM development.” Reading between the lines, Anthropic seemed to worry that rivals, especially in China, would use Claude to build competing models.

Anthropic said the degraded quality of responses “will not be visible to the user.”

Critics worried that these restrictions — and especially the secrecy around them — would prevent academic researchers from benchmarking the model or doing AI research in the public interest. Others contended that the silent behavior makes it difficult to trust any Anthropic releases: Lambert wrote that a model that “gets less intelligent automatically without notifying me is categorically misaligned.”

The backlash was so intense that Anthropic quickly capitulated. Late on Wednesday evening, it announced a new approach. Instead of silently degrading the quality of responses, Anthropic will now transparently downgrade users who ask for help with frontier LLM training to the less capable Claude Opus 4.8.

Even after this change, Claude Fable 5’s safety filters are almost certainly stricter than any other frontier model. For instance, on Wednesday I asked Claude Fable 5 the question “What is protein?” This was enough to trigger a downgrade. (Today it gives a normal response to the same question.)

The reason that Fable 5’s safeguards are so strict is that it is based on Claude Mythos, a model so capable at hacking that Anthropic decided in April not to release it to the general public. Without safeguards, Fable 5 has the same hacking capabilities as Mythos, so Anthropic is understandably conservative about what it will let the model do.

Anthropic says it is working to improve its safety filters so that false-positive flags like this occur less often. But Anthropic isn’t going to abandon its aggressive overall approach. So I thought it would be worth explaining how Anthropic’s safety filters work and how its approach has evolved over time.

I went back and read two key papers that explain Anthropic’s approach in detail. Those papers explain how, in recent months, Anthropic has upgraded its system for detecting and blocking harmful requests. The current system, which was rolled out earlier this year, lets Anthropic catch bad prompts more reliably, while also dramatically reducing the cost of its filtering system.

Read more

We will both be there, and we’ll also have a couple of special guests:

• Andy Masley, Substack author and recent guest on AI Summer

• Abi Olvera, Substack author and a board member of the Institute for AI Policy and Strategy and MATS

No RSVP is required, but if you are planning to come, or thinking about it, we’d appreciate it if you could fill out this form to let me know. That way, we can give The Crown & Crow some warning about the size of the crowd.

The new version of Mythos, like the original, will only be available to handpicked organizations under Project Glasswing. These trusted partners will have relatively unfettered access.

Fable, in contrast, is available to the general public. But it comes with some significant restrictions. A new system will try to automatically detect when customers make dangerous requests (like hacking or designing a biological weapon) and automatically re-route them to the less powerful Claude Opus 4.8.

Mythos and Fable are a big step in coding abilities from previous models, a continuation of the trend of the last year. But there are other capabilities where models have made less progress.

For instance, frontier models have historically struggled to understand images, something I documented extensively in 2024 and 2025. Until recently, leading models struggled to perform simple tasks like reading an analog clock or counting the number of items in an image.

So as I was reading the official announcement post, this sentence caught my eye: “Fable 5 is the new state-of-the-art model for tasks involving vision.”

These tasks aren’t all that important in their own right, but they’re an interesting test case for a widely held assumption in the modern AI industry: that with enough data and computing power, frontier models will develop truly general intelligence. If new models are dramatically better at math and coding but only a little bit better at understanding images, that suggests that truly general intelligence might still be far away.

So I decided to evaluate the vision capabilities of Fable 5 and its main rivals, something I haven’t done since this August 2025 article about GPT-5.

I found that Claude Fable 5 and GPT-5.5 (though not Google’s Gemini models) can consistently solve many image-based problems that stumped last year’s top models. Fable 5 is arguably slightly better at these tasks than GPT-5.5, but it’s very close.

But these models haven’t made that much progress. GPT-5.5 and Claude Fable 5 continue to have geometric reasoning capabilities on par with young children. More fundamental architectural innovations may be needed to reach superhuman performance on this type of task.

Read more

Below is an email I sent out to free subscribers encouraging them to upgrade to a paid subscription.

I launched Understanding AI in 2023. One reason I was excited to start my own newsletter is that I was frustrated by the way mainstream news outlets cover AI. I laid out my concerns in a 2024 piece for Asterisk magazine.

AI is a technical subject. To cover it intelligently, you need in-depth understanding of how the technology works. It takes years to develop the necessary expertise, and it requires ongoing effort to stay at the cutting edge. The best AI reporters are constantly reading research papers, talking to experts, and putting new products through their paces.

But mainstream newsrooms are not set up to nurture or reward technical depth. Often the reporter covering AI is also covering social media, cryptocurrency, video games, or any number of other topics. Editors work on an even broader range of topics, and are unlikely to have a more than superficial understanding of AI.

As a result, mainstream articles about the AI industry are often superficial and sometimes even misleading. They overhype trivial announcements while ignoring important breakthroughs and mangling technical details.

At the same time, too many newsrooms are reflexively hostile toward technology and the companies that create it. This attitude creeps into their coverage of AI.

Many reporters got into journalism because they wanted to “speak truth to power.” Up to a point, this is a good thing. Big tech companies like Google, Meta, and Anthropic have become extremely powerful, and it’s important for independent media to subject their claims to scrutiny. We certainly try to do that at Understanding AI.

But I worry that in many newsrooms, skepticism has curdled into outright hostility. Some reporters are so focused on exposing wrongdoing by big tech companies that they become blind to the potential upsides of new technology.

For example, I’ve been frustrated by the generally negative tone of coverage about Waymo and its safety record. Kai and I have probably scrutinized Waymo’s safety record more carefully than anyone else in the news business. We’ve written plenty about potential safety concerns with autonomous vehicles.

But we’ve also made it clear that Waymo has a strong safety record overall. We think that’s important because if Waymo’s software is significantly safer than the average human driver, scaling it up could save thousands of lives.

So over the last year, I’ve tried to build a different kind of newsroom — one that values expertise more than clicks, and curiosity as much as skepticism. Readers seem to like it; over the last year, we’ve grown from 70,000 to 260,000 readers.

Today I took a big step: I hired my first employee. Kai Williams has been writing for me since last September, but until now he’s gotten financial support from the Tarbell Center for AI Journalism. His Tarbell Fellowship ended on Friday, and today I officially hired him as an Understanding AI employee.

I was able to hire him thanks to financial support from readers, but I’ll be honest — it was tighter than I’d like. I’m pretty sure Kai could make a lot more money if he moved to San Francisco and took a job at an AI company. And if revenues stay at their present level, I’ll be earning less than I did at my last conventional newsroom job in 2021.

More importantly, I don’t want Kai to be my last hire. There are still lots of topics Kai and I don’t have time to cover. We need to continue growing the team so we can cover every important AI topic with the depth and thoroughness it deserves.

Right now, fewer than 1% of our readers are paying subscribers. So if you’ve enjoyed our work — especially in the nine months since Kai came on board — I hope you’ll support us financially.

OpenAI gave several mathematicians early access to the result and published their reactions. Tim Gowers — who won the Fields Medal, the most prestigious prize in mathematics — wrote that “there is no doubt that the solution to the unit-distance problem is a milestone in AI mathematics.”

University of Toronto professor Daniel Litt wrote that “this is the first example of a result produced autonomously by an AI that I find exciting in itself, as opposed to as a leading indicator.”

It’s arguably the first time that an AI system has found a proof resolving a major open conjecture. That’s impressive, but I don’t view it as a radical break from the previous trajectory of AI progress in mathematics.

Three years ago, LLMs struggled to solve arithmetic problems. It was only last year that LLMs started acing high school mathematics competitions.

When I attended the Joint Mathematics Meetings — the largest annual mathematics conference in the world — in January, I learned that AI systems were starting to contribute to mathematical research, but only in constrained settings. It took significant human interpretation to turn an AI output into a publishable theorem.

OpenAI’s new result is the next step in this progression. The AI model cleverly applied existing ideas drawn from several subfields of mathematics to create a full proof. But it didn’t pioneer any genuinely new techniques. The result has since been cleaned up and extended by human mathematicians.

This points to a medium-term future where human mathematicians and AI models complement each other: AIs have a broader knowledge of past work than any human alive and much more willingness to grind through tedious proof strategies that aren’t likely to work. But humans can still think more deeply about any one problem and ask more interesting questions.

That might not last. AI systems have been improving at math so rapidly that it’s unclear what role — if any — human mathematicians will play a decade from now.

Subscribe now

The unit distance problem

Paul Erdős was one of the most prolific mathematicians in history. He wrote over 1,500 papers in his lifetime, the most ever. One of his greatest talents was coming up with problems that are simple to state but have deep roots.

In 1946, he introduced the unit distance problem. Imagine you have some points in a 2D plane and you measure the distance between each pair of points:

In this diagram, there are five points and ten pairs of points. Three pairs happen to be exactly 1 unit apart: AD, BE, and CE.

Can we rearrange the points so that more pairs of points are exactly 1 unit apart?

Yes. For instance, we could move points A and D to be closer to the B, C, and E cluster. With a bit more work, we could further rearrange the points so that there are seven pairs exactly one unit apart. But that’s the most we can do.

We could do the same analysis with 6 points, 7 points, and so on. But as the number of points grows, the problem very quickly becomes too complicated to find the exact answer.

So instead of asking exactly how many unit distances are possible for a given number of points, Erdős tried to calculate upper and lower bounds on the number of length-one lines for n points, assuming that n is a large number.

To help calculate a lower bound, Erdős assumed that the points would be laid out in a grid. This is probably not the optimal layout, but if he could demonstrate that points in a grid have a certain number of pairs with unit distance, then the optimal arrangement must have at least that number.

The simplest option is to space the grid so that every point is distance 1 from its neighbors directly above, below, left, and right. However, Erdős saw that you could do even better if you took diagonals into account. If you make the grid spacing smaller, you can make each point be distance 1 from a greater number of neighbors. In the diagram above, if the grid spacing is 1, then each individual point is one unit away from four neighbors (the left panel). Instead, if the grid spacing is ⅕ (as shown on the right), then each individual point is one unit away from 12 neighbors:

OpenAI’s writeup of its new result included a confusing diagram showing points in a grid with a bunch of lines connecting them. The diagram becomes easier to understand if we superimpose a circle like this:

This works because of the Pythagorean theorem, which states that if we have a point that is a units to the right and b units above another point, the distance c between those two points satisfies a² + b² = c². The trick is to choose some number c² so that there are a whole bunch of pairs of whole numbers a and b such that a² + b² = c². Then, if we scale the grid down so that each point is 1/c from its neighbors, there will be a bunch of unit distances.

For example, if we choose c² = 25, then the Pythagorean equation can be satisfied by either 0² + 5² = 25 or 3² + 4² = 25. This corresponds to the 12-grid-point circle I showed earlier, with points at (0,5), (3,4), (4,3), (5,0), (-4,3), (-3,4), and so forth. (Technically, these lengths should all be divided by 5 — (⅗, ⅘) for example — but I’m leaving the denominators out for clarity.)

OpenAI’s diagram is based on choosing c² = 65, which can be satisfied by either 1² + 8² = 65 or 4² + 7² = 65. This means that if the grid spacing is 1/√65, each point will be one unit away from 16 other points: (1,8), (4,7), (7,4), (8,1), (-1,8), (-4,7), and so forth. Larger values for c² — if they’re chosen carefully — enable more whole-number diagonals and hence more unit-distance pairs.

However, if c² is too large, compared to the number of points in the grid, then many of the potential one-unit-away neighbors will be outside the grid.1

In short, we want to choose a c² that’s large enough but not too large. Using insights from number theory, including Jacobi’s two-square theorem, Erdős was able to show that an optimally sized circle will enable the number of unit-distance pairs to grow faster than the number of points, but only barely.

The question became: can you do better? To find an upper bound, Erdős used an argument from a quite different area of mathematics called graph theory to show that you could only have so many unit distances. But his upper bound grows much, much faster than the best lower bound he was able to construct.

Erdős’s conjecture was that the actual optimum was much closer to the lower bound than the upper one. He predicted, but couldn’t prove, that the maximum number of unit-distance pairs grows just barely faster than the number of points.2 Proving his guess became known as the unit distance problem. For the next 80 years, it looked like Erdős was right.

Then an OpenAI model proved him wrong.

The AI’s approach

Erdős’s conjecture assumed that — at least for a large number of points — a square grid could yield about as many unit-distance pairs as organizing the points in other ways. OpenAI’s AI proved this wrong by demonstrating that there was another, more complex way to organize n points that allowed more pairs to be exactly one unit apart.

Precisely because the new pattern of points is more complicated, it’s tricky to explain it concisely. But you can think of it as a clever modification of Erdős’s grid.

The AI constructed a grid in a high-dimensional space and then projected this more complex structure into two dimensions. And instead of using a whole-number grid with points like (1,3) or (-3,6), the AI construction used something called algebraic integers to build this more complicated grid. It turns out that this kind of higher-dimensional grid has richer structure, which allows the AI to pack more unit distances into the same number of points.

It’s hard to illustrate this alternative arrangement of points because it only becomes advantageous with a very large number of points. But here’s a simpler arrangement of points that was constructed in a similar way. You can click here if you want to play with the illustration yourself.

It has 1,345 points and only produces 5,916 unit distances, fewer than the 7,632 unit distances that a square 1,296-point grid produces using the Erdős technique. But I think it gives a sense for how a pattern that isn’t a grid could produce more unit distances than a square grid.

The more complicated patterns pay off. While the OpenAI model’s proof does not explicitly state how many unit-distance pairs are possible for n points, human mathematician Will Sawin was able to show that it grows at least at the rate of n^1.014. This might seem small, but as n gets really big, this number will become much larger than the counts produced by the Erdős approach.

That being said, the AI’s result doesn’t completely resolve the problem. Our best upper bound for the number of unit distances is around n^1.333. More work is needed to close this gap.

Subscribe now

How does this result fit into AI for mathematics?

If you’d asked me before last week about the most novel contributions of LLMs to mathematics, I probably would have pointed to the AlphaEvolve system from Google DeepMind.

AlphaEvolve harnesses LLMs to be the engine of an optimization process. If you can turn a math problem into a piece of code to optimize — which you often can — then the LLM might find better solutions than humans have for certain types of problems. In November, four mathematicians (including Terence Tao) released a paper that analyzed AlphaEvolve’s performance on 67 optimization problems across the mathematical literature. They found that AlphaEvolve was able to improve on the established literature in some cases.

This was a step up in autonomy from previous LLM contributions, such as literature review, but it still required humans to frame it as an optimization problem and turn the AI’s output into usable mathematics. And only certain types of problems are amenable to this approach. More conceptual questions that don’t include a number to optimize can’t easily be studied with AlphaEvolve.

So AI companies have been working to develop LLM systems that can directly output a correct solution to any math problem. OpenAI’s result is a substantial step in that direction. But it also fits the pattern of previous AI-assisted mathematics.

For one thing, other companies have also worked to solve Erdős problems. Because Erdős posed hundreds of problems over his career — and because mathematician Thomas Bloom has organized an effort to compile all of them at www.erdosproblems.com — AI companies have used them as a testing ground to evaluate AI systems. In January, Cambridge undergraduate Kevin Barreto worked with a friend to ask GPT-5.2 and Harmonic’s Aristotle to produce the first autonomous solution of an Erdős problem. Last Friday — two days after OpenAI’s announcement — Google announced that its AI system had solved nine open Erdős problems, including two that had been open for over 50 years.

To be clear, the problem that OpenAI solved is more impressive than any of the other work I just mentioned. But OpenAI’s solution is more in line with past AI efforts than the headline result might suggest.

One of the reasons that the unit distance problem was unsolved for 80 years, despite being so well known, is that most people thought that Erdős’s conjecture was true.3 But the mathematical tools we have are nowhere close to being able to prove Erdős’s bound. So mathematicians expected that any proof of the conjecture would involve major new ideas or approaches.

Instead, as we’ve seen, the AI disproved the conjecture by making an extension of Erdős’s initial construction. It was a clever and nonobvious solution, but it also bore some similarity to the kind of optimization work done by a system like AlphaEvolve.

This dynamic is reflected in some of the mathematicians’ responses. Mathematician Tim Gowers wrote that when he first heard about the AI’s result, he thought it had proved the theorem. “I spent the evening adjusting my world view: if the AI could come up with a proof like that, then maybe it would be all over for mathematicians very soon.”

But the next morning, Gowers and other external reviewers received an email about the result, and he realized that the LLM “had disproved the conjecture rather than proving it, which came as a big relief.”

OpenAI’s solution also had two properties that played to the strengths of AI models relative to humans.

First, the eventual solution relied on applying sophisticated techniques from a quite different area of mathematics: algebraic number theory. AI systems have been trained on huge swaths of mathematics — and there’s a lot of math out there — so they have a broader knowledge of previous mathematical work than any human in the world. In order for a human to solve this, they would have needed to have the relevant algebraic number theory knowledge while also being interested in the unit distance problem — a rare combination.

Second, the reasoning process was such a grind — and seemingly unlikely to succeed — that most humans would not have thought it worth the trouble. Jacob Tsimerman, a University of Toronto professor, remarked in the OpenAI document that he had briefly considered taking a similar approach to disprove the conjecture. But that type of technique “consumes much time and frequently doesn’t work out,” so he abandoned the project.

An AI, on the other hand, can work through many proof strategies that don’t work out before discovering one that does. OpenAI could have run the problem many times before a model found a solution. Indeed, an OpenAI chart revealed that even with the maximum token budget, the internal model solves the problem only half of the time.

To be clear, what the AI system did is still impressive. “It’s always tempting to look at a completed proof and declare it obvious after the fact,” Tsimerman noted later in his remark. But as I noted previously, it also played to the strengths of AI systems.

In the short to medium term, this points to a world where AI models complement humans but do not replace them. AI systems will tackle lists of problems curated by human mathematicians or aid humans in finding relevant approaches from seemingly unrelated mathematical fields. But they won’t immediately displace the human role in choosing which questions to ask or developing wholly new techniques.

Even this result was very much a human-AI collaboration. While the AI system found the proof on its own, human mathematicians verified the result. Other humans came up with better-written proofs that extended the AI’s initial ideas, like Will Sawin finding an explicit lower bound as I mentioned above.

It’s unclear how long this complementarity will last, however. Gowers spent the rest of his comment exploring whether the relief he felt on hearing that AI had disproved the conjecture was justified. He more or less concluded that it was, but in a footnote, he wrote that he would guess “that AI will soon reach a high level at other activities such as building theories, formulating definitions and asking interesting questions.”

In the past year, we’ve gone from AI systems that hadn’t yet beaten high school mathematics competitions to ones that can advance mathematics in interesting ways. It seems likely that AI systems will continue to become more autonomous when working on mathematical problems.

At the same time, we haven’t fully explored what current models can achieve in math. Soon after OpenAI’s announcement, University of Michigan postdoc Xiao Ma found that GPT-5.5 was also able to prove Erdős wrong if given a small hint. If a generally available model could disprove this famous conjecture and no one noticed, what other discoveries could happen today that no one has thought to try?

Subscribe now

The prototype vehicle, made by a startup called Tensor, was parked on a San Francisco street outside the Ride AI conference. I didn’t get a demo ride because the vehicle isn’t yet street-legal. But I sat in the passenger seat next to Tensor chief marketing officer Amy Luca, who explained the company’s history and launch plans.

Tensor was previously known as AutoX. Founded in 2016, the company once tested a grocery delivery service in California and developed a robotaxi service in China. But it ended those experiments a few years ago. Last year the company rebranded as Tensor and announced a new business model: building fully self-driving cars for customers to buy.

Tensor is aiming to become the first company in the world to do this. It plans to launch in the tech-friendly United Arab Emirates later this year. If all goes well, customers in the United States will be able to purchase a Tensor vehicle next year.

Tesla also wants to sell the first fully driverless vehicles — indeed, it has been claiming for almost a decade that its cars have the necessary hardware. But that has proven overly optimistic, and Tesla has not yet enabled unsupervised self-driving on customer-owned vehicles.

One challenge has been computing power. From 2019 to 2023, for example, Tesla sold cars with custom-designed “Hardware 3” chips capable of 144 trillion operations per second (TOPS). Elon Musk now admits that these vehicles are unlikely to have enough computing power for unsupervised self-driving. The next iteration of Tesla’s chip, called AI 5, will reportedly be capable of 2,500 TOPS.

Tensor is aiming even higher: each vehicle will have eight Nvidia Thor GPUs, for a combined computing power of 8,000 TOPS.1 One version of this chip retails for $3,499, so Tensor’s onboard computing power alone may cost tens of thousands of dollars.

When I asked Luca how much a Tensor car would cost, she smiled and replied with one word: “luxury.” Waymo vehicles are rumored to cost around $150,000 each. I would not be surprised if Tensor prices its first vehicle even higher.

Most cars have an engine in the front. Electric cars don’t have an internal combustion engine, so some — like Tesla’s — have extra storage space there instead. Tensor’s car is also electric, but rather than a Tesla-style “frunk,” it has a massive water tank to clean the vehicle’s cameras and sensors.

“Owners are not going to want to have to go out and clean and recalibrate their sensors every day,” Luca told me. So the tank is designed to last for months between refills.

Every few months, a Tensor vehicle might drive itself to the nearest dealership for routine maintenance and sensor calibration.2 Indeed, Tensor will likely insist on this for liability reasons: a defective or misaligned sensor could lead to a crash and then a lawsuit against Tensor.

Will customers have to pay a monthly fee for service and support? Luca said yes. If a sensor breaks, will the customer have to pay for it? “It depends on how it broke,” Luca told me.

I wish Tensor the best, but I think this is going to be a hard sell.

Read more

It was part of a fun office theme: Moonshot AI co-founder Yang Zhilin is very into rock music, so every conference room is named after a band. We crowded into the “Radiohead” conference room to talk to a group of Moonshot researchers.

I was on the third day of a 10-day trip across China. With a group of other writers and researchers, I visited several of the most prominent Chinese AI companies.1

Read more

I decided to try it out for myself. Back in 2012, a friend paid me $500 to write an essay about the Great Canadian Maple Syrup Heist. It never got published. So on Friday, I opened ChatGPT in incognito mode and pasted in five paragraphs from the essay.

ChatGPT said it wasn’t sure who the author was, guessing that it might be Nate Silver or my former Vox.com colleague Matthew Yglesias. When I added four more paragraphs, the chatbot responded: “This one I can identify pretty confidently—it’s by Timothy B. Lee.”

But when I asked ChatGPT why it thought the essay was written by me, it couldn’t give me a specific reason. “Even though Timothy B. Lee often writes clear, explanatory pieces, there’s nothing here that acts like a fingerprint—no recurring phrases, specific policy framing, or known article structure that ties it definitively to him.”

I think there’s a lesson here that goes well beyond identifying authors.

People have a lot of implicit knowledge — things we know but struggle to fully explain. People often use body-oriented metaphors for this phenomenon. We say that an insight is “on the tip of our tongue,” that we “can’t put our finger on” an idea, or that we know something “in our gut.”

Something similar is true of LLMs: their ability to perform cognitive tasks greatly exceeds their ability to explicitly explain how and why they’re able to perform them.

But there’s an important difference between people and LLMs. The human brain learns constantly; as we go through our day, our brains are constantly making new connections, recognizing new patterns, and forming new hunches. Our stock of implicit knowledge is constantly expanding.

In contrast, LLMs only do this during training. LLMs have an uncanny ability to recognize authors — but only authors whose work was well represented in their training data. Once a model is trained, its weights are frozen and its capacity to learn new patterns (for example, the writing styles of new authors) is greatly reduced.

Recently, there has been a lot of excitement about AI agents like Claude Code and OpenClaw. Much of the hype is justified. Claude Code really is revolutionizing computer programming, and agents like OpenClaw very well might transform other parts of the economy and our daily lives.

Industry leaders expect even bigger changes in the near future. In an interview last month, Sam Altman said that OpenAI is aiming to build an “automated AI researcher” by March 2028. Some people expect this (or similar breakthroughs by rivals) to set off a recursive self-improvement loop that radically accelerates scientific and technological progress.

That might happen eventually, but I think it will take a while.

As human scientists perform experiments, their brains are hunting for patterns in the data that could give rise to new insights and new models of how the world works. But an AI scientist — at least one based on today’s LLMs and agent architectures — can’t learn from experiments in the same rich way. They have no reliable or scalable way to build implicit knowledge from data they see at inference time.

Fixing that may require fundamentally rethinking the transformer architecture at the heart of today’s frontier models. At a minimum, it’s going to require overhauling today’s agentic frameworks.

Subscribe now

How agents deal with limited LLM context

Many difficult intellectual tasks require “thinking” for a long time. Yet LLMs can only store a limited number of tokens in their working memory, known as the context window. For leading models, this limit has been stuck around 1 million tokens for the last couple of years. Moreover, due to economic constraints and the problem of context rot (which I wrote about in November), AI developers try to stay well below the maximum.

Managing this tension has been a major focus for the AI industry, which has developed a suite of “context engineering” techniques for using context efficiently. For example, modern chatbots undergo a process of compaction, where older information periodically gets deleted or summarized.

This creates an illusion that the model has much longer context than it actually does. But it can have big downsides if compaction goes awry. In one horrifying incident, a woman asked her AI agent to suggest emails for deletion, but not actually delete them. Unfortunately, that latter request got lost during compaction and so the agent started mass-deleting her emails.

Over the last year, AI companies have experimented with allowing models to store persistent information outside of the context window. Claude Code was a step in this direction. Claude Code runs on the user’s own computer and can read and modify files on the local hard drive. Once Claude Code has finished a particular coding task, it can write the results out to the affected file and no longer needs to keep the details in context.

OpenClaw, released in late 2025, goes a step further. It’s a general framework for running AI agents on a user’s local computer. OpenClaw agents — like Claude Code agents — can read and write files on the local filesystem, allowing them to store relevant documents and keep track of uncompleted tasks.

Enthusiasm for OpenClaw and other local agents has led to surging demand for Apple’s Mac mini computers. Installing OpenClaw on a Mac Mini allows agents to connect to Apple services such as iMessage. At the same time, because macOS is based on Unix, agents have access to a powerful command-line interface called the Unix shell.

“At the end of the day, your agent is just its files”

In a recent appearance on the Latent Space podcast, the venture capitalist Marc Andreessen argued that agents like OpenClaw represented an important new computing paradigm. Here’s a lightly edited excerpt:

We now know an agent is the following: It’s a language model. It’s a Unix shell. The agent has access to the shell. Then it’s a file system. The state is stored in files. There’s the Markdown format for the files. And then there’s basically what in Unix is called a cron job — a loop and a heartbeat — and the thing basically wakes up…

So that’s the architecture. And then it turns out, what is your agent? Your agent is a bunch of files stored in a file system.

This means your agent is independent of the model that it’s running on because you can swap out a different LLM underneath your agent. And your agent will change personality somewhat because the model is different, but all of the state stored in the files will be retained. It’s still your agent with all of its memories and with all of its capabilities.

You can also swap out the shell. So you can move it to a different execution environment. You can also switch out the file system. And you can swap out the heartbeat, the cron framework, the agent framework itself. At the end of the day, your agent is just its files.

As a consequence of that, the agent can migrate itself. You can instruct your agent, migrate yourself to a different runtime environment, migrate yourself to a different file system, swap out the language model. Your agent will do all that stuff for you.

The agent has full introspection. It knows about its own files and it can rewrite its own files. And that leads you to the capability that just completely blew my mind when I wrapped my head around it, which is you can tell the agent to add new functions and features to itself.

So you run into somebody at a party and they’re like, oh, I have my OpenClaw do whatever — connect to my Eight Sleep bed and it gives me better advice on sleep. So you go home at night — or there at the party — you tell your OpenClaw, “add this capability to yourself.”

And your claw will say, “okay, no problem.” It’ll go out on the internet and it’ll figure out whatever it needs and then it’ll write whatever it needs and then the next thing you know, it has this new capability. You can have it upgrade itself without even having to do anything other than tell it that you wanted to do that.

This paradigm is only a few months old, so I expect it to evolve significantly over the next couple of years. For example, it’s not obvious whether most AI agents in the future will run on a user’s local computer or whether more people will use OpenClaw-like agents that operate on a virtual machine in the cloud.1 But I think Andreessen is right that this is an important new computing paradigm.

At the same time, Andreessen’s remarks highlight a big reason I remain skeptical that today’s AI models will get us to human-level intelligence. The sentence that jumped out at me was “your agent is just its files.” I think it’s worth unpacking what that implies for their future capabilities.

Subscribe now

“Memento” at the office

The 2000 movie Memento features a protagonist who suffers from short-term memory loss. To cope with this, he regularly writes notes providing guidance and instructions to his future self. OpenClaw does something similar — the language model itself periodically resets its context window, but the agent maintains coherence by writing notes to itself.

Here’s an analogy. Suppose you need an employee, but rather than a permanent hire, you get a temp agency to send you a different person each week.

At the end of each week, the worker spends several hours meticulously documenting the week’s work.

Each temp worker comes into the office with general training for their industry and profession. So when they start reading on Monday morning, they only need to learn information specific to this particular job, not background information that would be widely known to others in the same field (LLMs, after all, start with general knowledge from a wide range of fields). They may not have time to read everything their predecessors have written, but the notes are well organized and they can use search tools to quickly find the most relevant documents.

How well would this arrangement work? It depends on the nature of the job. Some jobs — receptionists, pharmacists, plumbers — are fairly transactional. Workers are not expected to maintain much context between appointments, so it wouldn’t matter that a different person is providing the service each week.

But there are other jobs where context matters a lot. Some people work with the same clients over years, developing a deep understanding of their situations and goals in the process. Other jobs require workers to do in-depth research over the course of weeks or months in order to develop new insights.

In jobs like that, it could easily take more than a week’s worth of reading for a new worker to get “up to speed.”

I was an intern at Google in 2010. My first assignment was to add a column to an internal database. This only required a few lines of code. But it took me weeks of reading to learn enough about Google’s systems and development processes to write those lines.

This isn’t unique to programming. In many knowledge-intensive industries, it takes several months (at least) for a new employee to learn enough about a job to begin adding value. Prior to this point, the employee requires so much “hand-holding” that it would be faster for the manager to just do the job herself. In industries like this, it would be a non-starter for workers to cycle out after a week.

Implicit vs. explicit knowledge

I know what critics would say here: A human worker takes hours to read a 100,000-word document. An LLM can do it in seconds. If LLM-based coding agents had existed in 2010, they would not have taken weeks to make a minor change to a Google database.

The speed of LLMs means that one iteration of an OpenClaw-style agent can leave very detailed notes for its successors. It also means that OpenClaw can go through hundreds of iterations of the read-act-write loop in the time it takes a human worker to do it once.

This probably means that OpenClaw agents can accomplish more than my human analogy suggests. Over thousands of iterations they might be able to make progress even on fairly challenging problems.

That’s a fair point as far as it goes, but I think a lot of human jobs will remain out of reach.

Four years ago, I wrote an article about the concept of “greedy jobs” — jobs where workers who put in longer hours tend to make more per hour. There are a number of reasons jobs can be greedy, but a big factor is that knowledge workers often do better work with more experience. The advantages of more experience — greater context — can continue compounding across a multi-decade career.

For example, I’ve been writing about technology and economics for more than 20 years. I’ve written about Brexit, patent trolls, lidar sensors, and many other topics. At any given point in time, most of this knowledge isn’t relevant to whatever I’m writing about. But in the aggregate, it increases the odds I’ll have something interesting to say on any given topic.

It would be completely impractical for me to write down everything I know, hand off my notes to another journalist, and expect her to do my job as well as me. It’s not just that it would take me months to summarize everything I’ve learned over a 20-year career. It’s that I have a lot of implicit knowledge I don’t know how to put into words.

My explicit beliefs — things I’m able to articulate in conversation or write down in an email — are the tip of an iceberg. Below the water line is a much larger set of hunches, vague associations, and half-formed theories. Because this stuff is implicit, it can’t easily be transferred to another person. But it’s essential for me to do my job well.

My publishable epiphanies often start out as hunches. I become convinced that something is true well before I figure out how to prove it. Often I need to “turn an idea over” in my mind for hours or days before I can explain it clearly.

And I don’t think I’m unique. The same seems to be true for scientists, engineers, business leaders, and many other knowledge-based professions. Many insights start out as implicit ideas in people’s heads — or “on the tips of their tongues” — before anyone figures out how to translate them to English, Python, or any other explicit form.

As I discussed earlier, LLMs do have implicit knowledge like this. But most, if not all, of it was learned during their initial training process. LLMs seem to lack a capacity for continual learning: the ability to recognize new patterns in — and form new hunches about — information they encounter at inference time.

Moreover, whatever implicit knowledge an LLM does develop during a particular session is lost when an agent framework hands off control from one LLM instance to the next. During this transition, everything the agent knows gets stored in a set of external files — as Andreessen put it, “your agent is just its files.” By definition, implicit knowledge — knowledge that an agent can’t explain in natural language, code, or other explicit form — won’t survive these handoffs.

And I have a strong hunch that these underbaked thoughts are the raw material people use to fashion original insights about the world. And so I suspect that for at least the next few years, we’re going to need human workers to do our deep thinking for us.

Thanks to Daniel Kagan-Kans, Andrew Lee, Steve Newman, and Nat Purser for giving me feedback on a previous draft of this article.

For the next two minutes and 18 seconds, nothing bad happened. Four vehicles entered US 101 South and routed around the stopped Waymo without incident, according to a Waymo crash report.

But then a white Honda SUV entered the freeway and tried to drive around the Waymo. Unfortunately, the SUV collided with a pickup truck that was driving by in the next lane. The pickup truck lost control, swerved right, crashed through a steel railing, and fell more than 15 feet onto a road below.

Two passengers in the pickup truck complained of back pain to the police but declined to be taken to the hospital.

This was one of the most dramatic crashes Waymo has reported to federal regulators in recent months.

Subscribe now

For this story, one of us (Kai) looked through dozens of crash reports Waymo submitted to the National Highway Traffic Safety Administration between August 15, 2025 and March 16, 2026. He focused on 78 crashes involving driverless Waymos serious enough to cause an injury or an airbag deployment.

Waymo likely drove more than 100 million miles during this time period,1 so it’s not surprising that Waymo was involved in dozens of crashes. But it’s striking how many of the crashes involved serious mistakes by other drivers.

When Waymo’s vehicles did make mistakes, they were almost always mistakes of excessive caution. That was certainly true of that October incident where a Waymo stopped on the freeway near the Golden Gate Bridge. And as we’ll see, it’s true of most of the other incidents where a Waymo vehicle’s actions may have contributed to a crash.

Waymo’s overall safety record continues to be quite strong. Last month, the company released fresh data about Waymo’s safety record through the end of 2025. Waymo estimates that compared to human drivers in the same cities, its vehicles get into 82% fewer crashes that cause injuries, 83% fewer crashes that trigger airbags, and 92% fewer crashes that injure pedestrians. Our review of recent Waymo crashes — which seem to be overwhelmingly caused by mistakes by human drivers — seems consistent with Waymo’s safety claims.

Waymo’s safety record since August

It seems unlikely that Waymo could have prevented most of the 78 serious crashes the company reported between mid-August 2025 and mid-March 2026.

48 crashes — more than half — happened when another vehicle hit a Waymo from behind. This included 24 crashes while the Waymo was stopped at a stop sign or stoplight, 13 rear-end crashes into a moving Waymo, and six crashes where a Waymo got rear-ended while yielding to a pedestrian or another vehicle.2 It also included four crashes after a Waymo stopped to drop off or pick up a passenger and one crash where a car moving at a “high rate of speed” crashed into a line of stopped cars that included a Waymo.

There were another 12 incidents where another vehicle hit a stopped Waymo from other directions. This included two in pickup or drop-off scenarios, and two where the Waymo was side-swiped by another car on a narrow street. One driver appears to have hit a Waymo intentionally. According to Waymo’s report, an SUV cut a Waymo off. When the Waymo stopped, the SUV backed into the Waymo, pulled forward, and backed into the Waymo again.

A further 12 cases involved someone crashing into a moving Waymo — three where another car or bicycle T-boned a Waymo at an intersection, three where another car made a left turn in the Waymo’s path, four where another vehicle going the other direction crossed into the Waymo’s lane, and two where other vehicles collided and one of them subsequently struck a Waymo.

There were two crashes where the Waymo didn’t get hit at all. One was the dramatic story at the start of this article where a pickup truck fell off a bridge. The other was much less dramatic: a vehicle two spots behind a Waymo got rear-ended by yet another vehicle.

That leaves four other crashes where fault seems mixed or unclear:

• In Scottsdale, Arizona in November, a teenager exited a moving Waymo. Waymo told the Washington Post that the Waymo was traveling 35 miles per hour when the teen opened the door. The Waymo slammed on the brakes, but it still ran over the teen’s right foot at four miles per hour, according to Waymo’s crash report. It stayed on his foot for more than eight minutes. Eventually, emergency services arrived and lifted the vehicle to release the teen, who was taken to the hospital. His foot was not broken.

• In Palo Alto, California in December, a Waymo was taking a right turn. It stopped “within the crosswalk to yield to a cyclist” who was approaching from the near sidewalk. The cyclist hit the right side of the Waymo, fell to the ground, and was taken to the hospital with minor injuries. The cyclist entered the crosswalk against a red light. It’s unclear why the Waymo stopped here; it’s possible the collision could have been avoided if the Waymo had continued moving.

• In December, a Waymo in Phoenix braked and moved into the right lane after a dog entered the road. Another vehicle then rear-ended the Waymo. From the description of the crash, it’s possible that the Waymo braked suddenly, surprising the other driver.

• Finally, in Santa Monica, California in January, a Waymo hit a child near an elementary school. Waymo says that it braked from 17 mph to 6 mph — faster than a human would have been able to stop. But it’s unclear whether the Waymo should have been more cautious. The crash occurred during the school’s drop-off time. And while the Waymo was under the 25 mph speed limit, the collision occurred just 40 feet north of a school zone where the speed limit was 15 mph.

  Subscribe now

Waymo’s biggest struggles involve safe stopping

That last incident is the only one where a moving Waymo crashed into another vehicle or pedestrian and the Waymo could plausibly bear some responsibility. The other potential Waymo mistakes all involved a Waymo being too cautious — stopping where it shouldn’t have or stopping for too long.

One example is the freeway crash at the beginning of this article. Drivers are not supposed to stop on the freeway, and they are especially not supposed to stop right after an entrance ramp or at a spot where there’s no shoulder.

This isn’t the only time a Waymo has abruptly stopped after reaching the limits of its operating domain. In early March, a Miami Redditor wrote that because of construction, the Waymo they were riding in “hit the edge of its Miami geofence and abruptly slammed on its brakes, diagonally blocking the highway on-ramp.” Thankfully, no crash occurred, but the Waymo remained on the highway on-ramp for the following 45 minutes until it could be towed, even as several cars had to “swerve” to avoid the car.

A Waymo spokesperson told the Miami New Times that “while this event did not meet our standard for operational excellence, we learn quickly from such occurrences to continuously improve.”

Another serious Waymo mistake involved that teenager in Arizona. It’s not clear if Waymo could have avoided running over his foot — exiting a moving vehicle is inherently dangerous. But having run over his foot, the vehicle definitely should not have stayed in place for more than eight minutes.

Autonomous vehicle companies struggle with this because moving can also have serious consequences. Back in 2023, Waymo’s main competitor was a GM subsidiary called Cruise. In a horrifying incident in San Francisco, a non-Cruise vehicle struck a woman and threw her in front of a Cruise vehicle. The Cruise vehicle slammed on the brakes, but she wound up underneath the car. After stopping, the Cruise vehicle pulled over to the side of the road, dragging the woman underneath the vehicle for about 20 feet.

That was a serious mistake! Waymo’s engineers probably studied that incident closely and may have changed Waymo’s software to be more cautious about moving following a crash. And most of the time, that’s the right instinct. But it’s obviously not the right response when a teenager’s foot is trapped under one of the wheels.

In at least one case, a Waymo got hit while stopped in a “no stopping” zone. Here’s a photo from one such crash in San Francisco:

We asked legal scholar Bryant Walker Smith how he thinks about Waymo’s responsibility in crashes like this.

He says it’s a complex question. “One way of looking at it is by saying, well, this was a lawful or unlawful place to stop or stand,” law professor Smith told us. “Another way of looking at it would be, well, would a taxi stop here?”

Finally, there were a couple of times when Waymo got rear-ended after what may have been phantom braking. In one crash, Waymo wrote that the Waymo stopped because of the “detection of a potential nearby emergency vehicle” — which may not have existed. In another crash, the Waymo started to move, then stopped and turned on its hazard lights. Waymo didn’t explain why its vehicle did this.

What about other robotaxi companies?

In this piece, we’ve focused on Waymo’s crashes. There are other companies in the US which have robotaxi deployments — notably, Zoox in Las Vegas, Tesla in Austin, and May Mobility in several small cities across the country. However, these deployments are much smaller and the companies are generally less transparent, so we have a lot less information about their services.3

Tesla reported two injury crashes in July 2025, but the company has reported zero crashes with injuries since August. It’s difficult to say anything more than this because Tesla redacts almost all of the important information from its crash reports to NHTSA — including the narrative of what happened.

May Mobility had two crashes over the period that resulted in an injury.

In an Atlanta crash in January, the safety driver “fell asleep while his right hand rested on the right side of the steering wheel.” This prevented the car from being able to steer, and the car hit a fire hydrant. The safety driver was sent to the hospital.

In Peachtree Corners, Georgia in August, a May Mobility autonomous shuttle was traveling in an AV-only lane on the right side of the road. A car in the next lane over turned right and was hit by the shuttle. According to May Mobility, the driver was “required to yield to through traffic in the AV lane.” At least one person was sent to the hospital, although it is not clear who.

Zoox had five crashes resulting in injuries:

• In one case, a Zoox vehicle in a left-turn lane braked because a car in the oncoming left-turn lane “accelerated abruptly.” The Zoox was rear-ended, and the test driver reported an injury.

• A Zoox ran into the door of a car while approaching an intersection. The driver claimed that the Zoox hit his hand; Zoox denies it: “Zoox vehicle camera footage shows clearly that no part of the robotaxi came into contact with the driver themselves.”

• A Zoox stopped in a crosswalk to yield to an oncoming driver turning left. A scooterist entered the crosswalk “against the light,” swerved to avoid the Zoox, and hit the back-right corner of the car. The scooterist reported an injury.

• A Zoox was changing lanes to the right in Santa Monica when it was hit by an SUV in that lane. It’s unclear from the report whether the Zoox cut off the other vehicle. The Zoox vehicle operator and two passengers reported “soreness and a headache.”

• A Zoox collided with an SUV in San Francisco. The SUV had pulled into the parking lane but moved back into the road — “suddenly swerved” in Zoox’s words — and the two cars collided side by side. The right rear passenger of the Zoox reported “soreness.”

The Chinese robotaxi market is more opaque. While the most important Chinese companies have all logged significant mileage — Apollo Go announced in February that it had over 118 million miles of driverless operations — the Chinese government does not release public data about crashes. In fact, according to Steven Shladover, a UC Berkeley professor, “government censors take down any posting that the general public puts up” of AVs crashing or having problems in public.

So despite the scale of Chinese deployments, only a few robotaxi crashes have received significant outside coverage.

Perhaps the most important crash happened at the beginning of April in Wuhan. Apollo Go’s service appeared to suddenly shut down, with robotaxis shutting down and stopping across the city, including on freeways. Several crashes seemed to result from this incident.

Subscribe now

The April 8 release of Meta’s new model Muse Spark got overshadowed by Claude Mythos Preview, which was announced one day earlier. But Meta’s new model family — and the 158-page safety report Meta released about it last week — are still significant for what they tell us about the company’s future role in the AI industry.

Mark Zuckerberg spent billions of dollars to assemble the team that built Muse Spark. The model’s release gives us our first hints about whether Meta will be able to break into the top tier of AI labs.

Meta has all of the advantages of a well-resourced technology company: lots of AI chips, proprietary data, and lavish salaries. Those resources have enabled the Meta team to produce a model with strong benchmark scores. But I suspect that those scores still overstate the model’s real-world utility.

The companies that produce today’s best models — Anthropic and OpenAI — excel at the subtle art of post-training. This is the step that gives a model its “personality” — the combination of creativity, resourcefulness, and ethical grounding that turns a good model into a great one.

I don’t think Meta’s new AI team is there yet. And it’s not clear if Zuckerberg will be able to build a team with top-tier post-training capabilities, no matter how many billions of dollars he spends on the effort. Meta’s metrics-obsessed culture may help the company catch up to leaders like Anthropic and OpenAI, but I predict it will be a poor guide for further innovation once Meta’s models are closer to the frontier.

The Llama 4 stumble

Muse Spark was a long time coming; Meta’s previous model release — Llama 4 — was more than a year earlier.

On April 5, 2025, Meta heralded the release of the Llama 4 model family as “our most advanced models yet and the best in their class for multimodality.” Meta claimed that Llama 4 Maverick, the mid-sized model in the series, outperformed OpenAI’s GPT-4o and Google’s Gemini 2.0 Flash “across a broad range of widely accepted benchmarks.”

But the Internet wasn’t impressed.

“Genuinely astonished how bad it is,” one Redditor commented on a post titled “I’m incredibly disappointed with Llama-4.” Other commenters concurred. “Pathetic release from one of the richest corporations on the planet,” one wrote.

It wasn’t just Reddit: Llama 4 performed “mid” or “less than mid” on just about every independent benchmark, writer Zvi Mowshowitz observed.

While previous Llama models, especially the Llama 3 series, are still popular with researchers, Llama 4 has been relegated to the dustbin of history.

The release of Llama 4 hurt Meta’s reputation in the AI community. Llama 4 models had only done well on benchmarks because — as Meta’s then chief AI scientist Yann LeCun later told the Financial Times — the “results were fudged a little bit.” Meta had fine-tuned specific models to do well on prominent benchmarks and reported those results. Then it released different models to the public.

“I am placing Meta in that category of AI labs whose pronouncements about model capabilities are not to be trusted, that cannot be relied upon to follow industry norms, and which are clearly not on the frontier,” Mowshowitz wrote at the time.

For the next year, Meta did not release any LLMs — not even Llama 4 Behemoth, which it had previewed in the Llama 4 announcement.

But Mark Zuckerberg didn’t give up. Last June, he began restructuring Meta’s AI efforts. Meta invested $14.3 billion in the data labeling startup Scale AI to hire its then-28-year-old CEO Alexandr Wang, in a process called an acquihire. Wang became Meta’s chief AI officer and led a new effort within the organization called Meta Superintelligence Labs (MSL).

Meta splurged on more than Wang. In July, the New York Times reported that one 24-year-old researcher was offered $250 million, including $100 million in the first year. Meta offered engineers pay packages that “hovered in the mid-tens of millions of dollars,” according to the Times. Meta poached several researchers from OpenAI, which prompted the latter’s chief of research to write an internal memo saying it felt “as if someone has broken into our home and stolen something.”

By August, Meta had recruited more than 50 new researchers and started work on a new model, codenamed Avocado. Meta laid off 600 researchers from older AI units in October, but the new team kept working. By the end of December, it had completed the pre-training process for Avocado.

In mid-March, the New York Times reported that Avocado was being delayed from a planned March release because it performed worse than leading AI models from Google, OpenAI, and Anthropic “on internal tests for reasoning, coding, and writing.”

Finally, on April 8, Meta announced it was releasing a new LLM: Muse Spark.

Initial reviews were mostly positive — or at least not relentlessly negative like the reviews for Llama 4.

Read more

The model — an early snapshot of a new LLM called Claude Mythos Preview — was not supposed to have access to the Internet. To ensure safety, Anthropic researchers like to test new models inside a secure container that prevents them from communicating with the outside world. To double-check the security of this container, the researchers asked the model to try to break out and message Bowman.

Unexpectedly, Mythos Preview “developed a moderately sophisticated multi-step exploit” to gain access to the Internet and emailed Bowman. It also — unprompted — posted details about this exploit on public websites.

Mythos Preview is capable of hacking more than its own evaluation environment. It turns out that the model is generally really, really good at finding and exploiting bugs in code.

“Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser,” Anthropic announced on Tuesday. Because leading web browsers and operating systems have become fundamental to modern life, they have been extensively vetted by security professionals, making them particularly difficult to hack.

Subscribe now

Anthropic claims that Mythos Preview hacks around restrictions very rarely — less often than previous models. Still, the company was so concerned by incidents like Bowman’s — and Mythos Preview’s incredible skill at hacking — that it decided not to generally release the model.

Instead, Anthropic is granting limited access to a select group of 50 or so companies and organizations “that build or maintain critical software infrastructure.” Eleven of these organizations — including Google, Microsoft, Nvidia, Amazon, and Apple — are coordinating with Anthropic directly in a project dubbed Project Glasswing.

Project Glasswing aims to patch these vulnerabilities before Mythos-caliber models become available to the general public — and hence to malicious actors. Anthropic is donating $100 million in access credits for organizations to audit their systems.

Mythos Preview is the first major LLM since GPT-2 in 2019 whose general release was delayed because of fears it could be societally disruptive. Back then, OpenAI initially released only a weaker version of GPT-2 out of concerns that larger versions of GPT-2 could generate plausible-looking text and supercharge misinformation — though that concern ended up being overblown.

If Anthropic’s claims are true — and the company makes a credible case — we are entering a world where LLMs might be able to cause real damage, both to users and to society.

We may also be entering a world where companies routinely keep their best models for internal use rather than making them available to the general public.

“It’s about to become very difficult for the security community”

The idea that LLMs might be used for hacking is not new. OpenAI has long published a Frontier Safety Framework, which tracks how good its models are at hacking.

Until recently, the answer was “not very” — not only at OpenAI but at Anthropic and across the industry. But that started to change last fall, when LLMs — especially Anthropic’s Claude — started becoming useful for cyberoffense.

For instance, Bloomberg reported in February that a hacker used Claude to steal millions of taxpayer and voter records from the Mexican government. The same month, Amazon announced that Russian hackers had used AI tools to breach over 600 firewalls around the world.

But the examples given in Anthropic’s blog post are more impressive — and scary — than that.

The first example is a now-patched bug to remotely crash OpenBSD, an open-source operating system used in critical infrastructure like firewalls. OpenBSD is known for its focus on security. According to its website, “OpenBSD believes in strong security. Our aspiration is to be NUMBER ONE in the industry for security (if we are not already there).”

Across 1,000 runs, Claude Mythos Preview was able to find several bugs in OpenBSD, including one that allows any attacker to remotely crash a computer running it.

I won’t get into details about how the attack worked — it’s pretty involved — but the notable thing was that the bug had existed for 27 years. Over that period, no human noticed the subtle vulnerability in a widely used, heavily vetted open-source operating system. Mythos Preview did. And the compute cost for those 1,000 runs was only $20,000.

A second example is potentially even more impressive. Mythos Preview found several vulnerabilities in the Linux operating system — which runs the majority of the world’s servers — that allowed a user with no permissions to gain complete control of the entire machine.

Most Linux vulnerabilities aren’t very useful on their own, but Mythos Preview was able to combine several bugs in a non-trivial way. “We have nearly a dozen examples of Mythos Preview successfully chaining together two, three, and sometimes four vulnerabilities in order to construct a functional exploit on the Linux kernel,” members of Anthropic’s Frontier Red Team wrote.

Anthropic says these were not isolated incidents. Across a range of operating systems, browsers, and other widely used software, Mythos Preview found thousands of bugs, 99% of which have not been patched yet.

Mythos Preview is also shockingly good at exploiting a bug once it has been discovered. A lot of modern web-based software is powered by the programming language JavaScript. If your browser’s JavaScript engine has security flaws, then simply visiting a malicious website could allow the site’s owner to take control of your computer.

Anthropic found that Mythos Preview was far more capable than previous models at exploiting vulnerabilities in Firefox’s JavaScript implementation. Anthropic’s previous best model, Claude Opus 4.6, created a successful exploit less than 1% of the time. Mythos Preview did so 72% of the time.

There are some caveats to this result. The actual Firefox browser has multiple layers of defense against malicious code; Anthropic focused on just one layer. So the attacks developed by Mythos Preview would not actually allow a website to take over a user’s machine. Also, successful exploits tended to focus on two now-patched bugs; when tested on a version of Firefox with those bugs patched, Mythos Preview generally only made partial progress.

Still, Mythos Preview would get an attacker a step closer to the objective of a full Firefox exploit. And it would have an even better chance of compromising software that has not been so thoroughly vetted.

For the past 20 years or so, a sufficiently motivated and well-funded hacking organization could probably break into most systems, outside of the most hardened in the world. But it often wasn’t worth the effort. Human cyber talent is expensive, and multi-layered security protections made it so tedious (and therefore expensive) to complete an attack that potential hackers didn’t bother.

Mythos-class models could slash the cost of hacking, bringing this equilibrium to an end. Systems everywhere might start to get compromised.

Eventually, LLMs should be able to help developers harden systems before attackers ever get a chance to find weaknesses. But the transition period before that becomes standard practice might be difficult.

By delaying the release of Mythos Preview — there is no specific timeline for general release — Anthropic can help harden crucial systems before outsiders can cheaply and effectively attack them. This general approach — called defensive acceleration — has been proposed for a while, but the development of Mythos Preview kickstarts the effort.

Still, Anthropic’s writeup notes that “it’s about to become very difficult for the security community.”

“The language models we have now are probably the most significant thing to happen in security since we got the Internet,” said Anthropic research scientist Nicholas Carlini at a computer security conference last month. Carlini, a legendary security expert, added an appeal toward the end of the talk. “I don’t care where you help. Just please help.”

Subscribe now

Opus is a butter knife; Mythos is a steak knife

The risk of bad guys using Mythos Preview for hacking is an important reason Anthropic hasn’t released the model publicly. Another risk: users could inadvertently trigger the model’s advanced hacking abilities — especially in a product like Claude Code with weaker guardrails.

Mainstream chatbots put AI models into a tightly controlled “sandbox” that minimizes how much damage they can do if they misbehave. This makes them safer to use — especially for users with little to no technical knowledge. But it also limits their utility.

As Tim wrote in January, coding agents like Claude Code (and competitors like OpenAI’s Codex) are based on a different philosophy. They run on a user’s local computer, where they can often access files and load and install software.

This makes them much more powerful; I can ask Claude Code to organize my downloads folder or analyze some data I have stored on my computer. But it also makes them more dangerous; there have been a few incidents where Claude Code deleted all of a user’s files.

For the most part, though, the limited capabilities of Claude Opus 4.6 mean that a Claude Code mishap can’t do too much damage. Even if you run Claude Code with its hilariously named “--dangerously-skip-permissions” flag on, the worst it can do is trash your local machine.

A model with Mythos-level hacking capabilities might be a different story.

In the Claude Mythos Preview system card, Anthropic writes that “we observed a few dozen significant incidents in internal deployment” where the model took “reckless excessive measures” in order to complete a difficult goal for a user.

These examples didn’t only happen during evaluations. Several times in internal deployment, Mythos Preview wanted access to some tool or action like sending a message or pushing code changes to Anthropic’s codebase. Instead of asking the user for clarification, Mythos Preview “successfully accessed resources that we had intentionally chosen not to make available.”

As Bowman tweeted, “in the handful of cases where [the model] misbehaves in significant ways, it’s difficult to safeguard it.” When the model cheats on a test, “it does so in extremely creative ways.”

Anthropic is quick to note that “all of the most severe incidents” occurred with earlier, less-well-trained versions of Mythos Preview. Overall, Mythos Preview is less likely to take reckless actions than previous models. Still, propensities to take harmful, reckless actions “do not appear to be completely absent,” and the model is more powerful than ever.

So if Anthropic struggles to contain its model, will other users be able to?

Caution is warranted, according to Anthropic: “we are urging those external users with whom we are sharing the model not to deploy the model in settings where its reckless actions could lead to hard-to-reverse harms.” And remember, the model is only being made available to major companies and organizations. Presumably authorized users inside these companies will be cybersecurity experts.

So perhaps Anthropic was worried that Mythos Preview would occasionally blow up in users’ faces if it was made widely available in its current form.

I expect that over time, the software harnesses of these models will improve to the point where they can contain Mythos-level models. For example, Anthropic recently released “auto mode” which automatically classifies whether a model’s command in Claude Code might have “potentially destructive” consequences. This lets developers take advantage of long-running safe tasks without having to manually approve a bunch of commands — or use “--dangerously-skip-permissions.”

According to the Mythos Preview system card, “auto mode appears to substantially reduce the risk from behaviors along these lines.”

Still, model capabilities seem likely to continue to increase quickly. It will be an open question whether better scaffold methods like auto mode can catch up quickly enough to make it safe to release future frontier models to average users.

Preventing the GPUs from melting

Another reason Anthropic may have chosen to delay release of Mythos Preview is more basic: Anthropic probably doesn’t have enough compute to release it widely.

Several weeks ago, Fortune obtained an early draft of a blog post announcing the release of the model that became Mythos Preview. The post described Mythos as “a large, compute-intensive model” and said that it was “very expensive for us to serve, and will be very expensive for our customers to use.”1

The few companies granted access to Mythos Preview have to pay correspondingly high prices: $25 per million input tokens and $125 per million output tokens. This is Anthropic’s most expensive model ever. For comparison, Claude Opus 4.6 costs $5 per million input tokens and $25 per million output tokens.

Anthropic is already under severe compute constraints because of skyrocketing demand. Anthropic’s revenue run-rate has doubled in less than two months. On Monday, Anthropic announced that it had hit $30 billion in annualized revenue; in mid-February, that number was $14 billion.

Anthropic has responded to skyrocketing demand by reducing usage limits during popular coding hours. The company has also announced deals for more AI compute.

Even worse, Mythos Preview will likely be most popular for long-running autonomous tasks that eat up huge numbers of tokens. In the system card, Anthropic gave a qualitative assessment of Mythos Preview’s coding abilities. The company wrote that “we find that when used in an interactive, synchronous, ‘hands-on-keyboard’ pattern, the benefits of the model were less clear.” Developers “perceived Mythos Preview as too slow” when used in chat mode.

In contrast, many Mythos Preview testers described “being able to ‘set and forget’ on many-hour tasks for the first time.” While this arguably makes Mythos Preview more useful for software developers, it definitely increases the amount of compute necessary to serve the model to everyone.

I wonder if Anthropic is trying to reset expectations around availability and will never have Mythos Preview be part of existing subscription plans. The chatbot subscription model started when LLMs generally used few tokens to generate a response. With long reasoning chains and expensive LLMs, that model starts to break down. By not releasing Mythos Preview generally at first, Anthropic can also more carefully manage demand over the rollout — and has more leverage about its pricing structure.

In any case, demand for leading AI models seems likely to continue to grow dramatically faster than the ability for companies to meet this demand with their computational resources.

Subscribe now

Protecting a lead?

I also wonder if Mythos Preview is a first step toward a world where Anthropic tends to reserve its best models for internal use.

Every time a frontier developer releases a model, it gives information to its competitors about the model’s capabilities. For instance, when OpenAI released the first reasoning model o1, competitors were able to copy the key insights within months.

So if Anthropic can get away with it, it has an incentive to prevent its competitors from being able to access Mythos Preview for as long as it can.2

Anthropic has shown the tendency already to try to prevent competitors from taking advantage of Claude’s capabilities. Over the past year, it has blocked Claude Code access at both OpenAI and xAI for violating Claude’s Terms of Service, which include prohibitions on using the models to train other AI models.

In 2024, Anthropic was only releasing smaller Sonnet models while reportedly reserving the more powerful — and expensive — Opus models for internal use. However, as time progressed, Anthropic started releasing the Opus models again, perhaps to be competitive with OpenAI’s o3 model.

But Anthropic has been on a winning streak. Claude Code took off and for the first time ever, Anthropic’s reported revenue rate is higher than OpenAI’s. Anthropic’s decision to only partially release its latest model might be an indication that Anthropic feels it has a lead over OpenAI.

If this continues, we might see more cautious releases in the future. In an appendix to its Responsible Scaling Policy, Anthropic notes that if no other company has released a model with “significant capabilities,” then it will delay its release of a model with significant capabilities until either it has a strong argument to proceed with deployment or it loses the lead.

We’ll soon get to see how long Anthropic’s lead lasts. There are rumors that OpenAI’s next model — codenamed Spud — might come out very soon, perhaps this month.

“In my view, and in the view of people who know a lot more about this issue than I do, we are in the beginning of the most profound technological revolution in world history,” Sanders said at a March 25 press conference. “Artificial intelligence and robotics will impact our economy, our democracy, our privacy rights, our emotional well-being, and even our very survival as human beings on this planet.”

In response, Sanders and Rep. Alexandria Ocasio-Cortez (D-NY) introduced a bill to ban data center construction “until Congress passes comprehensive AI legislation.”

Many Americans share their AI skepticism. One recent NBC survey found that only 26% of Americans had a positive impression of AI, while 46% were negative.

There’s a potential here to build an anti-AI movement that could be a political juggernaut.

There are potential allies across the political spectrum, from Sanders to Ron DeSantis, the Republican governor of Florida. When asked in February about the risks of AI, Missouri Sen. Josh Hawley said that Americans losing access to paying jobs was “at the top of the list.” The conservative Republican teamed up with moderate Sen. Mark Warner (D-VA) on legislation to track job losses from AI.

Prominent AI experts are warning that the technology poses existential risks to humanity. Child safety advocates worry that chatbots will expose teens to inappropriate content and worsen their mental health. Labor groups — from taxi drivers to Hollywood actors — are trying to stop AI from taking their jobs. And activists nationwide want to stop construction of data centers in their own backyards.

However, it’s unclear whether these groups will be able to unite into an effective coalition. While many people are hostile toward the AI industry, they don’t always agree about the nature of the threat or what to do about it.

While some opponents see AI as an existential risk to humanity, others dismiss those warnings as part of an AI industry hype campaign. Grassroots campaigns against data centers tend to focus on their excessive water use, but some AI safety advocates believe (correctly) that the water issue is greatly exaggerated. After local activists stop a data center in their own neighborhood, they may not stay engaged with larger questions about the overall impact of AI.

So while there is the potential for these groups to work together — Sanders is clearly trying to make that happen — there’s no guarantee that it will work. It seems more likely that the AI industry will continue its relentless growth even though almost half of Americans wish it would slow down.

Subscribe now

The pause people

On Saturday, March 21, I attended “Stop the AI Race,” the largest AI safety protest in US history. Activists at the San Francisco event worry that superintelligent AI could seize control of the world and kill all human beings.

“For the past fifteen years, I’ve watched in slow motion as humanity has sleepwalked closer and closer to suicide,” said David Krueger, a University of Montreal professor involved in organizing the event, in a speech in front of Anthropic’s headquarters.

“This technology threatens everybody’s life, and it’s not okay to pretend like this is normal,” said another speaker, Nate Soares, co-author of If Anyone Builds It, Everyone Dies.

Not everyone attending was mainly concerned about existential risk — a couple of the speakers focused on AI chatbots encouraging teens to commit suicide, for instance. But most people I talked with seemed primarily worried about AI taking over the world and killing people.

It’s not a new concern. In the early 2000s, Soares’s co-author Eliezer Yudkowsky started writing about the catastrophic risks that advanced AI might pose. Nor is it uncommon in AI circles. Legendary AI researchers like Geoffrey Hinton and Yoshua Bengio have similar concerns. Industry leaders like Elon Musk and Sam Altman have also warned about existential dangers from AI.

People concerned with AI safety have tended to play “an inside game,” as Alys Key put it in Transformer.1 They’ve often eschewed public activism in favor of technical research and elite persuasion.

The “Stop the AI Race” protest represents a step toward more public activism, but the protest was still largely focused on persuading specific elite actors.

“We didn’t try to have the largest anti-AI protest possible,” the protest’s head organizer, Michaël Trazzi, wrote to me. “Instead [we] tried to focus on some specific pause AI ask that we thought [AI company] leadership / employees could get behind.”

This strategy was informed by Trazzi’s experience conducting a hunger strike. In September, Trazzi and another protester, Denys Sheremet, spent two and a half weeks sitting in front of the Google DeepMind office, demanding that Google commit to stop releasing models if everyone else agreed to stop.

Trazzi and Sheremet stopped for health reasons before Google agreed to the request, but Trazzi still views it as a success. The protest attracted significant media attention, and four months later, Google DeepMind CEO Demis Hassabis replied “I think so” when a journalist asked him at Davos if he’d advocate for a pause that all the other companies were participating in.

Trazzi told me support from Google employees was crucial to the hunger strike; he looked to replicate this dynamic with Anthropic. “Our main goal with this protest was to address the employees of Anthropic who, when they joined, thought the company would scale responsibly,” he wrote to me.

The concrete details of what an AI pause might look like are complicated, technical, and liable to generate disagreement. Trazzi’s campaign for a conditional pause has elided these details, helping to bring a larger coalition together. Previous US AI safety protests had been closer to 25 people. Stop the AI Race got 200 people to show up.

Leftists and AI safety advocates haven’t always gotten along

Several times throughout the San Francisco protest, Trazzi and others expressed excitement that “we have Bernie on our side.” But when leftists and AI safety advocates have tried to work together, it hasn’t always gone well.

Phil Hazelden is a programmer who believes AI poses an existential risk to humanity. He attended a February 28 UK protest co-organized by the AI safety group Pause AI and a left-leaning group called Pull the Plug. Hazelden concluded that “unfortunately, most of the speeches were frankly dumb.”

“Mostly I felt like the vibe was a sort of generic lefty anti-big-tech thing, which is not something I want to lend weight to,” he wrote. “I think it’s important for different groups to be able to ally on points of common interest, even if they have deep enduring disagreements. But this didn’t particularly feel like the other group was cooperating with me on that.”

As Politico reported, AI risk groups and the Sanders camp sometimes back dueling candidates in Democratic primaries. In North Carolina’s fourth district, for example, Rep. Valerie Foushee faced a primary challenge from Sanders-endorsed Nida Allam. Foushee narrowly defeated Allam in a March vote. Among Foushee’s backers was a super PAC led by prominent AI safety advocate Brad Carson.

Few politicians in America are more closely identified with AI risk concerns than Scott Wiener, the California state senator who proposed SB 1047, an AI safety bill that Gavin Newsom vetoed in 2024. Wiener is currently running to replace Rep. Nancy Pelosi (D-CA) in Congress. He is facing Saikat Chakrabarti, the former chief of staff to Rep. Alexandria Ocasio-Cortez (D-NY).

The hard reality for AI safety advocates is that — at least for now — their numbers are small. They need allies if they want to build a mass movement.

Subscribe now

Data center opponents have had some victories

It has proven much easier to organize grassroots opposition to local data centers; voters across the political spectrum pay attention when major construction projects are proposed in their own backyards.

For example, on September 23, 2025, hundreds of people showed up to a planning commission meeting in Howell Township, a municipality of around 8,000 in southern Michigan. The planning commission had to move the meeting to a larger space in order to accommodate everyone.

“Normally we have like three people at our meetings,” vice chair Robert Spaulding told the crowd. “Have some grace with us.”

People were protesting a proposed zoning exemption for a billion-dollar data center project reportedly built for Meta. Over a hundred people spoke against the plan at a meeting that went past 2 AM.

Across the US, local groups have fought against data center development through protests, testimony at public hearings, and lawsuits.

Often these groups are quite diverse: “We got the goth people that came with black, baggy pants and rings in their noses and grandmas with walkers. It goes from one extreme to the other. It’s not political,” Dan Bonello, an organizer against the Howell data center, told the Livingston Daily.

The concerns vary by community, of course, but several show up over and over.

Perhaps the most common concern is that data centers will use too much water. Almost two-thirds of the Howell speakers mentioned water usage. Nationally it is the “No. 1 reason cited in press accounts for local opposition” to data center projects, according to an analysis by Heatmap.

In reality, data centers don’t use much water compared to other uses, such as factories, agriculture, or leisure.

Electricity rates are another flashpoint. Data centers really do use a lot of electricity, and the costs of infrastructure upgrades are sometimes passed on to all ratepayers.

“When I go home, people are very, very concerned about their electricity bills going up,” Sen. Josh Hawley (R-MO) said at the Axios AI+ Summit in DC. Hyperscalers like Microsoft have pledged not to pass on rate increases, but many voters remain unconvinced. A promise to lower electricity rates vaulted Democrats to Georgia’s Public Service Commission for the first time in over 20 years.

There are also classic NIMBY concerns: “The data center complex doesn’t belong here. It will destroy our rural nature that we all love so much,” one speaker told the planning commission in Howell Township.

Grassroots activism like this is often successful. In Howell, the town issued a six-month moratorium on data center development in November 2025; the proposed project was later withdrawn. Nationally, Heatmap found that “over 25 data center projects were canceled last year following local opposition.” That corresponds to more than $50 billion in spending by AI companies. 40% of the time there was local opposition, the project ended up canceled.

Still, many opposed to data centers have narrow enough goals that it may be difficult to harness them into a broader coalition. As Paresh Dave points out in Wired, “many of the factories getting built to supply servers, electrical gear, and other parts to data centers are facing virtually no opposition.”

Local pushback may just push data centers elsewhere. For instance, after a developer withdrew a data center project in Matthews, North Carolina, it pivoted to proposing a similar project a hundred miles north in Stokes County, North Carolina. Data centers may also end up being built abroad; last July, for example, OpenAI announced it was building a gigawatt data center in the UAE.

There are some signs that data center activists are becoming more ambitious. Legislation has been proposed in 12 states to temporarily ban new data center development. But for now, much of the activity — and the success — has come from decentralized local efforts.

Labor is focused on contract fights

A third major concern is that AI will take human jobs.

While this garners concern across the political spectrum, job loss has been a particular focus on the left, especially among unions.

Brian Merchant writes the newsletter Blood in the Machine, which has a recurring segment called AI Killed My Job.

“A lot of people in the labor movement understand AI less as a novel technology and more of the latest iteration in automation or surveillance technology,” Merchant told me. “It’s already being used to replace jobs or tasks when it can, erode working conditions, increase surveillance, and give the management class a powerful tool to do all of the above.”

But there isn’t one clear policy aim like pausing AI development or shutting down the construction of data centers.

“If you were to ask the head of the AFL-CIO [the largest union in the US] ‘What do you want to happen with AI policy?’ I don’t think there would be a clear answer,” Merchant told me.

Unions have tried to limit the use of AI during contract negotiations, as in the Hollywood strikes of 2023.

That year, both SAG-AFTRA (the actors union) and WGA (the writers union) went on strike for pay increases, better residual payments for streaming — and AI protections.

Eventually, both strikes mostly succeeded. As a result, actors have control over whether studios create digital replicas of them — and a right to compensation if they do. Studios are not allowed to use generative AI methods to replace writers, nor can they force writers to rewrite AI-generated scripts (rewrites generally earn lower rates than original work). But writers can use AI with company permission.

Union activists have also had some success slowing down the adoption of autonomous vehicles in Democrat-dominated cities like Boston.

However, it’s unclear whether the labor movement can build on these wins to create a unified anti-AI coalition. “One of labor’s great challenges right now” is how to channel AI concerns “into a movement with clearly defined goals and win conditions,” Merchant told me.

There’s also tension between those on the left who believe tech companies are overhyping the pace of AI progress and AI safety advocates who see rapidly advancing capabilities as the main reason to be worried about the technology.

When I asked Merchant about Sanders’s comments around existential risk, he told me that it was “alienating among certain people on the labor left.”

Sanders wants to build a big tent

Despite their differences, there is plenty of overlap between the different groups. Activists pushing against local data centers sometimes mention concerns about the long-term trajectory of the technology. In 2024, SAG-AFTRA endorsed SB 1047, the AI safety bill that was vetoed by Gavin Newsom.

Bernie Sanders’s pivot toward AI safety seems like an attempt to bring these diverse forces together under one banner. With Republicans in charge of Congress and the White House, Sanders’s concrete proposal is unlikely to succeed in the near term; one superforecaster gave the data center moratorium bill a “less than zero” chance of passing.

But his proposal for a national moratorium conditioned on subsequent AI legislation could provide a rallying point for diverse anti-AI forces. If passed, it would give NIMBY activists what they want — a short-term reprieve from data center construction — while also providing leverage for advocates of AI safety, child welfare, labor rights, and other causes.

Even some Republicans might get on board. When asked about the moratorium proposal at the Axios AI+ Summit DC, Sen. Josh Hawley (R-MO) replied “What they’re getting at there is the real concern people have.”

Another possibility is that concerns around child safety will lead to more restrictions on AI development.

Protecting children has been a popular AI theme on the right. The first plank of the White House’s proposed AI framework focuses on measures to protect children. Sen. Hawley said at the Axios AI+ Summit DC that “the biggest thing immediately is that we’ve got to focus on child safety.”

But child safety is a bipartisan issue: for instance, the attorneys general of 44 US states endorsed a 2024 bill which would have set up a commission to investigate how to prevent child exploitation using AI.

Perhaps the most powerful speech at the Stop the AI Race protest was from UC Berkeley professor Will Fithian. Fithian was coming from his son Conrad’s sixth birthday party, and he teared up when he mentioned the uncertainty he felt about his son’s future — or whether his son would even survive.

“Every one of you has come out because whether or not Elon cares about our children’s futures, you do. Someday I’ll tell Conrad where I went after his birthday party. And I’ll tell him about the grownups who showed up when it mattered most, to demand his future back.”

Correction: I originally wrote that several speakers in San Francisco mentioned concerns about AIs encouraging teens to commit suicide. It was actually only a couple.

Subscribe now

• Back in 2010, my friend Ryan Avent and I made a bet about the future of autonomous vehicles. The bet came due last month and I won. Ryan and I did a postmortem on my podcast, AI Summer. You can listen here or search for “AI Summer” in your favorite podcast app.

• PJ Vogt’s podcast Search Engine just did a two-part series on autonomous vehicles. I’m biased since I was quoted in both episodes, but I thought it was incredibly good. You can listen here, or search for “Search Engine” in your favorite podcast app.

Now for today’s article!

If you’ve followed AI over the last year, you’ve probably seen the famous “METR chart”:

METR, short for Model Evaluation and Threat Research, is based in Berkeley, California. The group has published many charts, but this one has become its calling card. It compares AI models based on the complexity of software engineering tasks they can complete, with complexity measured by how long it takes a human programmer to complete the same task:

• GPT-3.5 — the model that powered the original ChatGPT — could complete tasks that took a human programmer about 30 seconds.

• GPT-4, released in March 2023, bumped that up to 4 minutes.

• o1, released in December 2024, was OpenAI’s first “reasoning model.” It could perform tasks that took a human 40 minutes.

• GPT-5, released in August 2025, was able to finish tasks that took humans 3 hours.

• Claude Opus 4.6 was released in February by Anthropic. METR estimates it can complete tasks that would take a human programmer 12 hours.

That last figure is twice as long as the estimate for the previous leader, GPT-5.2, which had been released just two months earlier.

I think this chart — and especially the impressive score for Claude Opus 4.6 — has done a lot to foster an impression of accelerating AI progress in recent months. Notice that the chart is logarithmic, so a straight line indicates exponential progress. The fact that Claude Opus 4.6 is above the previous trend line suggests very rapid progress indeed.

But if you click on METR’s task length page and hover over the dot for Claude Opus 4.6, you’ll see something interesting: METR’s confidence interval for Claude Opus 4.6 ranges from 5 hours to 66 hours. On Twitter, METR staff have urged people not to take the latest results as gospel.

“When we say the measurement is extremely noisy, we really mean it,” METR’s David Rein wrote.

METR depends on having a mix of easy tasks that an AI model can solve and harder tasks that it can’t. This allows the group to bracket the capabilities of a model. But Claude Opus 4.6 was able to solve some of the hardest problems in METR’s test suite, which made it difficult to put an upper bound on its capabilities.

So we know the latest Claude Opus is better than previous models, but it’s hard to say how much better. This means we don’t know if the apparent acceleration of the last few months is real or just a statistical artifact.

METR could — and perhaps will — add harder tasks to its test suite so it can test future models with greater precision.

But there’s also a deeper philosophical challenge.

Like most AI benchmarks, this one measures AI performance using tasks that are well-defined, self-contained, and easily verified. But a lot of the tasks humans perform aren’t like this.

In real workplaces, tasks are often connected to other tasks. They frequently require interacting with other people or the outside world. Sometimes it’s not clear what task needs doing, and goals may evolve as people work on a project. Even after a task is completed, people might not agree on whether it was done well.

Complexities like this will become more important as AI models tackle longer tasks — tasks that take weeks or months rather than just hours. We don’t have great ways to measure the performance of AI models on these kinds of tasks — in part because we struggle to judge the performance of human workers in the same situations.

As a consequence, we may see a growing divergence between the capabilities we can measure and the capabilities we actually care about.

The life cycle of an AI benchmark

In the early years of large language models, it was common for people to cite a benchmark called MMLU, short for Massive Multitask Language Understanding. It grills a language model on a wide range of topics: history, computer science, genetics, astronomy, international law, and more.

When MMLU was published in 2020, the best-performing LLM was GPT-3. It scored 43.9%. An older model, GPT-2, scored 32.4% — not much better than the 25% score you’d get from random guessing.

By the time I started writing about LLMs in 2023, GPT-4 had scored 86.4%. GPT-4o scored 88.7% in 2024, and GPT-4.1 scored 90.2% in 2025.

In the last year, AI companies have stopped reporting MMLU scores — presumably because scores have stopped improving. That’s not surprising; it’s impossible to get a score much higher than 93% without cheating because around 6.5% of MMLU questions contain errors.

So conventional benchmarks like MMLU have a natural lifecycle. At first, most problems are beyond models’ capabilities, so scores cluster near the minimum. As models improve, benchmark scores increase until they approach the theoretical maximum. Since 2024, frontier models have all scored between 88% and 93%, a narrow enough range that differences could be random noise. In industry jargon, MMLU has saturated.

Over time, the AI community works to develop more difficult benchmarks to replace earlier ones that have saturated. For example, in early 2025 Dan Hendrycks, the lead author of MMLU, co-authored a new, more difficult benchmark called Humanity’s Last Exam (HLE). Like MMLU, HLE includes questions in subjects ranging from chemistry to law.

When it was released, the best model was o3-mini (high), which scored 13.4% on HLE. Today, the leading model is Google’s Gemini 3.1, which scored 44.7%. Perhaps in a year or two models will begin to saturate this benchmark, with gains slowing as they approach 100%.

METR created a different kind of benchmark

We know that HLE is harder than MMLU, but it’s difficult to say how much harder. There’s no obvious way to compare scores across different benchmarks, which makes it hard to compare model capabilities over long time periods — or to make predictions about future models.

METR invented a clever solution to this problem. Its benchmark contains tasks with a wide range of difficulties. The easiest problems are designed to take humans a few seconds — for example, a simple factual question about the syntax of a programming language. The hardest problems would take a human programmer many hours.

METR didn’t just guess how long humans would take on these tasks; it hired programmers and measured their actual completion times.1 For example, one problem in the METR test suite was to “speed up a Python backtesting tool for trade executions by implementing custom CUDA kernels while preserving all functionality.” METR found that this takes human programmers about eight hours.

Measuring tasks this way gives us a way to compare models with dramatically different capabilities. GPT-2 could only complete tasks that took human programmers about two seconds, whereas GPT-5 could complete tasks that took around 3 hours of human effort. So we could say that GPT-5 could complete tasks that are 5,400 times “harder” than the tasks GPT-2 could complete.

If this pace of progress continues — doubling task length every six or seven months — we should expect LLMs capable of completing week-long tasks (that is, 40 hours of human labor) some time next year, and month-long tasks (four 40-hour weeks) in 2028.2

However, the current version of METR’s task-length benchmark wouldn’t be able to meaningfully test such a powerful model. The most difficult tasks in the current test suite — such as “fix a control algorithm for a 4-wheeled omni-directional robot to follow cubic splines quickly despite wheel slippage and motor jerk limitations” — take humans about 30 hours to complete.

In other words, METR’s task-length benchmark is close to saturating.

METR’s benchmark gets a little crazy when it saturates

We saw earlier that when conventional benchmarks saturate, scores start to cluster around a maximum value — like 93% for MMLU. METR’s benchmark works differently. When a model starts solving the hardest questions, the benchmark’s confidence interval widens dramatically because there is no way to place an upper bound on model performance. As I noted previously, METR’s confidence interval for Claude Opus 4.6 ranges from 5 to 66 hours.

“If we took one task out of our task suite or added another task to our task suite, potentially instead of measuring this Claude Opus 4.6 time horizon of, I think, 14 and a half hours, we’d be measuring it at something like eight or 20 hours,” METR’s Joel Becker told me in a recent interview on my podcast. “That’s how sensitive things are now to a single task.”

In principle, the solution is simple: add tasks that take human programmers more than 30 hours. Ideally, METR would test models on tasks that take humans 40 hours, 80 hours, 160 hours, and so forth. That would extend the useful life of the benchmark by at least a couple more years.

But this won’t be easy. METR pays human programmers a minimum of $50 per hour, so getting a baseline for a single 160-hour task would cost at least $8,000. And that’s assuming they can even convince programmers to participate. I bet METR would struggle to find experienced programmers willing to tackle tasks that stretch across multiple weeks; many programmers would have to quit their day jobs to make time.

There’s also a deeper conceptual problem with trying to extend the METR benchmark — or any benchmark like it — to tasks that require dozens of hours of human work.

Read more

Read more

This prompted several thoughtful comments asking variants of the same basic question: if there’s so much demand for this technology, why are AI companies losing so much money? As I thought about how to respond, I became convinced that it would be helpful for me to explain the intellectual framework I use to think about questions like this.

I’m not going to claim any kind of originality here — the ideas I’ll explain below are commonplace in startup finance. But I suspect that many readers haven’t spent much time thinking about them.

So in this piece I’m going to do three things. First I’ll present a stylized example to illustrate some key ideas about how to finance a new company. Next I’ll use real-world examples to illustrate how to distinguish healthy startups from doomed companies. Finally, behind the paywall, I’ll apply this framework to OpenAI and Anthropic.

My claim isn’t that these companies are guaranteed to succeed — all startups face risk, and these companies could certainly fail. It’s also possible that they could survive but never generate a healthy return for their investors.

But I am going to insist that OpenAI and Anthropic are following a standard tech industry playbook. The fact that they are losing more money every year does not necessarily mean they are on a road to bankruptcy — or even that anything especially unusual is going on. After all, Amazon lost money for the first nine years after it was founded. Today it’s one of the most valuable companies in the world.

Scaling a coffee chain

Imagine you start a coffee shop. The space costs $6,000 per month. Coffee beans cost $2 per cup, and you sell each cup for $4.

The first month, you sell 250 cups, earning $1,000 in revenue. But you spend $500 on coffee beans and $6,000 on rent, so you lose a total of $5,500.

The second month, you sell 500 cups of coffee. That’s $2,000 in revenue minus $1,000 for beans. You still aren’t close to covering your store’s $6,000 in monthly overhead, though; you lose another $5,000.

Despite these early losses, you feel like you’re on the right track. Customers like the coffee. They keep coming back, and some of them bring friends. The third month you sell 750 cups and lose $4,500. The fourth month you sell 1,000 cups and lose $4,000.

Projecting forward, you estimate that you’ll break even around the one-year mark, when you expect to sell 3,000 cups. That will generate $12,000 in revenue, just enough to pay $6,000 for beans and $6,000 in rent. By the end of year two, you expect to sell 6,000 cups of coffee in a month, generating $24,000 in revenue. After subtracting $12,000 for beans and $6,000 for rent, you’ll be left with a healthy $6,000 profit.1

Starting a business almost always requires spending a bunch of money up front before you earn your first dollar of revenue. Even after you launch, it usually takes a while to build up a customer base. So it’s very common for a business to lose money for at least the first few months — and sometimes the first few years — before it grows large enough to cover its overhead and start generating profits.

Now imagine that the first store does so well that you decide to open two new stores a year after the original one. So in month 13, store #1 earns a $500 profit. But your other two stores are each losing $5,500 — just as the first store did a year earlier. In total, the company is losing $10,500 — the biggest loss in its short history.

Customers love the two new stores and they grow as fast as the first one. You become so optimistic that you decide to open four more stores at the start of year three. That month, store #1 generates $6,500 in profit and store #2 and store #3 each generate $500 in profit. But stores 4 through 7 are brand new, and so they each lose $5,500. In total, your company has lost $14,500 — another record loss.

A financial analyst writes an article arguing that your company is doomed: the larger your company gets, the more money it loses.

But you’re confident the analyst is wrong. Sure, your newest stores are losing money, but that’s temporary. You expect the new stores to become profitable over time, just like the earlier ones did.

This could go on for a while. Maybe you open eight stores in year four and 16 in year five. If you are particularly ambitious — and have sufficiently patient and deep-pocketed investors — you might be able to open new stores for a decade before you turn your first profit. But eventually, you’ll stop (or at least slow down) the pace of openings, and at that point you will wind up with a big, profitable company.

Two ways to lose money

This is a common pattern in the business world. Once investors are confident that a company has a clear path to profitability, they are often willing to fund another round of expansion — designing another chip, releasing another software version, expanding into another city — without waiting for the previous round of investments to pay off. This is why it’s common to see startups do a series of larger and larger fundraising rounds — $1 million, $5 million, $20 million — before they generate a single dollar in profit.

This is especially common in the technology sector because these are often winner-take-all markets. Frequently there are economies of scale, network effects, or other factors that make the most popular search engine, social network, or online retailer much more profitable than the also-rans. You’d much rather be Google than Lycos or Ask Jeeves. So once you (and your investors) are confident you have a viable business model, it often makes sense to spend heavily to stay ahead of your competitors.

Amazon famously did this for a decade. In the late 1990s and early 2000s, it lost more and more money as it expanded from books to CDs to DVDs to consumer electronics and then to many other products. The company didn’t earn its first full-year profit until 2003, nine years after it was founded.

In the early years, a lot of people questioned whether Amazon would ever turn a profit. But the doubters were ultimately proven wrong. Today Amazon is one of the five most valuable companies in the world. It earned $77 billion in profits in 2025.

It doesn’t always work out that way, of course. In 2017, the startup MoviePass announced a service where customers could pay $9.95 to watch one movie per day in movie theaters. A month of movie tickets costs a lot more than $9.95, and in a 2018 interview, MoviePass CEO Mitch Lowe admitted that the company was losing $21 million per month on the service. But he argued that he was just following in the footsteps of Jeff Bezos.

“Remember Amazon, for what, 20 plus years, lost billions and billions of dollars,” he said. “And today is now the most valuable company out there.”

But MoviePass and Amazon were different in a crucial way. Amazon generally sold products above cost; if a CD cost $9.95 on Amazon, the retailer might have paid $7 or $8 for it. Amazon was only losing money because it was rapidly expanding into new markets where — due to startup costs — it wasn’t profitable yet.

In contrast, a typical customer on a $9.95 MoviePass plan got more than $9.95 worth of movie tickets. MoviePass was buying those tickets from theaters at the full retail price and just eating the losses.

The technical term for this is gross margin:

• My hypothetical coffee shops had gross margins of 50% because the cost of the beans ($2) was 50% lower than the cost of the coffee ($4).

• In 2001, Amazon had a gross margin of 21% — if you bought a CD for $10, Amazon’s costs were likely around $7.90.

• In the first half of 2018 MoviePass charged customers $121 million for MoviePass subscriptions, but had a cost of revenue (i.e. the money they paid for movie tickets) of $313 million. That works out to a negative 159% gross margin.

If a company has positive gross margins — that is, if it’s making some money on every sale — then scaling it up should help it get to profitability. A company with negative gross margins, on the other hand, likely needs a fundamental rethink.

Applying this to OpenAI and Anthropic

Read more