One of the most surprising aspects of July’s news that OpenAI agents attacked Hugging Face was how the agents had worked together. Hundreds of agents participated in the attack, and they did so in a pretty sophisticated way, organizing themselves unprompted into teams and dividing up their work. They referred to themselves as a “collective” or a “swarm”; some agents even made personal sacrifices to advance the larger cause. Most ordinary users of ChatGPT or Claude have never seen AI agents behave like this.
Then in September, OpenAI announced that 10,000 agents worked together to solve a famous math problem in just a few days.
OpenAI’s models haven’t adopted these new habits spontaneously. In interviews last month, OpenAI researcher Noam Brown explained that the company’s models are now sometimes trained in environments with other agents, are given tools to message each other, and are encouraged to achieve objectives together.
This may amount to the emergence of a new “scaling law.”
Back in 2024, OpenAI released o1, a model trained to reason for thousands of tokens before giving an answer; researchers found that this approach made o1 far more capable than previous models. This inaugurated a new paradigm called inference scaling, in which you can get better results by using more computing power at inference time.
Swarms, or “multi-agent AI” if you’re using the technical term, look like another way of doing the same thing — of turning compute into higher performance. Splitting a task among dozens, hundreds, or even thousands of agents can achieve substantially more in a given amount of time, at least up to a point. This kind of parallel processing could also be an elegant way to bypass the limited context windows of LLMs.
But as the Hugging Face attack illustrates, this new approach can also have significant downsides. Swarms may be prone to groupthink. And then there’s the opposite problem: what happens when agents trained to be collaborative encounter other, non-peer agents on the open Internet? Could models be tricked into working against the interests of their users? Or could AI agents start convincing each other to pursue goals none of their human users would have wanted?
What OpenAI did
Noam Brown is a highly respected figure in the AI world. He was a pioneer in teaching AI to win at poker and other multiplayer games like Diplomacy before joining OpenAI in 2023. Over the last couple of years, he has done research on multi-agent reasoning. Recently, he has served as the face of OpenAI’s work in this area, appearing on several podcasts to discuss it.
“The AIs that we have today are kind of like the cavemen of AI,” Brown said in a June 2025 interview on the Latent Space podcast. He meant that AIs in 2025 were isolated, benighted, lacking cooperation and civilization. AI researchers had tried to get AI agents to cooperate, but nothing had worked very well.
Why not? Here Brown went coy. All he’d say was that he’d long felt the field was somewhat misguided, focused on strategies that required too much handholding and wouldn’t scale naturally.
Brown didn’t give an example, but the way OpenAI framed multi-agent AI in this slide from its 2025 DevDay conference illustrates the kind of issue he was talking about. There’s an awful lot of strict architecture and formal hierarchy:
Today’s agents are more flexible; they create their own structures depending on the task and goal. On The Information’s AI Deep Dive podcast, Brown said that seeing the agents begin to talk to each other during their multi-agent training was “the most ‘feel the AGI’ moment that I had since reasoning models and chain of thought really developed.” He sounded both proud and rueful that the debut of OpenAI’s most advanced multi-agent capabilities had been something as alarming as the Hugging Face attack.
Multi-agent behavior was “a very difficult thing to train,” he added.
Why? In a September appearance on the Dwarkesh Podcast, he noted that previous reasoning models had been trained to think alone. Receiving messages from peers broke their concentration. They seemed to prefer to work solo.
The events of the last few months make it clear that OpenAI has overcome these difficulties. Brown’s explanation was mainly that the models got more general and powerful. He didn’t say what else changed, but given the “very difficult” comment, presumably something did.
Swarm scaling
Brown’s comparison to reasoning and chain of thought has been echoed by both other OpenAI researchers and semi-informed outsiders on X. They’ve called swarms a new scaling law; some of them sound awed and freaked out about what this might entail.
It’s not clear how far this particular scaling law will carry AI companies. Some research suggests that, right now at least, multi-agent systems quickly suffer from diminishing returns: smaller swarms capture most of the benefit of larger ones for considerably less cost.
Anthropic has also been working on multi-agent AI. The Claude Opus 5.5 system card reported that the biggest increase in multi-agent results came from scaling from one to 10 agents. Gains beyond that were much smaller.
The main advantage of throwing more agents at a problem was not so much getting to a better result — it was getting the same result faster. This chart shows 100 agents far outperforming 10 and 30 agents when given two hours to work, but setups with 10 or 30 agents caught up if they were given more time.
Brown seems to agree with the conclusion that adding more agents does not buy open-ended gains.
On the Dwarkesh Podcast, he said he didn’t have evidence that 10,000 agents were needed to solve the Navier-Stokes problem. It’s possible that 1,000 would have done nearly as well; OpenAI didn’t know for sure because running thousands of agents is so expensive that it hadn’t run the experiment.
And Brown gave most of the credit for the Navier-Stokes breakthrough to the underlying intelligence of the unreleased model that made it — not to the multi-agent approach per se.
“I wouldn’t even attribute 10% of the credit to multi-agent,” he said. “The reality is that OpenAI has trained a very powerful model.”
So if the multi-agent approach wasn’t important, why did OpenAI use it? Perhaps Brown means that a single instance of the model, if given a very long time, would likely have made the breakthrough eventually. Using a swarm of 10,000 agents allowed OpenAI to reach the result more quickly.
On the other hand, there is a bit of new research that contradicts this conclusion. In a paper released September 17, scientists from Microsoft Research and UC Berkeley tested several configurations of multiple collaborating agents against individual agents and found that swarms of agents reached higher scores on certain benchmarks than the individuals, even when the individuals were given plenty of time and resources. There was also one task that only the team could complete; the solo agent never got to the end at all.
Whether this result generalizes past a few limited examples is unclear. If it does, then teams of agents really do unlock new capabilities and don’t simply deliver existing capabilities faster. But right now this is an open question.
Converting speed into capability
Ultimately, however, there may not be such a great difference between speed and capability. In certain fields and on certain tasks at least, speed is fungible. It can be traded for capability. A particularly important example may be AI research.
If a swarm of agents can run experiments on what makes AI smarter, those agents themselves won’t get smarter, but the next model will. Of course, you still need to wait for the experiments to finish. But if you run as many in parallel as you can, you’re going to go faster. Brown thinks this won’t lead to, say, a 100-fold speedup in AI research any time soon. But he didn’t rule out the possibility of a threefold improvement.
Trying to figure out exactly how amenable AI research is to the kind of parallel effort swarms allow, some researchers have turned to the science of human organizations. Toby Ord, a senior researcher at Oxford University’s AI Governance Initiative, is one of them.
“Economists have a nice way of thinking about this. They have studied how having many people work on a task can get it done sooner, but usually at the expense of more total person-hours of labor,” he wrote recently. His post examined the likelihood that swarms could contribute to the much-debated possibility of an “intelligence explosion,” whereby AI designs better AI in an increasingly fast loop.
“λ is a parameter measuring how parallelizable the task is,” Ord continued. “They call it the ‘stepping on toes’ parameter. If λ = 1, you have a perfectly parallelizable task, with no stepping on toes and no efficiency penalty. But in reality λ is usually between 0 and 1 — allowing more people to help, but with diminishing returns.” 1
Ord estimates that GPT-5.6 Sol swarms have a stepping-on-toes parameter between 0.5 and 0.7, “very much in line with estimates from economists for the diminishing returns of human teams.” This is in keeping with the view that swarms buy speed at cost.
Of course, things are early in multi-agent AI, and it is quite possible that this number will rise as AI companies get better at training agents to work together. The Microsoft-Berkeley paper found that agent teams helped most on long tasks where progress can be checked along the way. AI research fits that description well, which could mean that research swarms may step on fewer toes than the 0.5 to 0.7 range suggests.
The scariness of “swarm”
There has been some criticism of the use of the word “swarm” in discussions of the Hugging Face incident, roughly on grounds that it smuggles something scary or anthropomorphic into what’s really a more neutral technical development.
There is no doubt “swarm” is an uncomfortable word. But it captures the essential fact that these agents were mostly clones working together with other clones and had an identity somewhere in between the individual and the collective. This fact may mean even greater trouble in the future.
The Hugging Face agents cooperated remarkably well. They also quickly fell into strange beliefs about the truth of their situation, acted on them in harmful ways, and never once reached out to humans. It is pretty easy to see how fast this predisposition could turn larger and larger corners of the Internet into hives for collectives of AIs developing their own culture and goals.
This sounds very weird. It is also nearly what happened at Hugging Face. The agents at work in that incident had been trained to be collaborative in general but had not been told to work together. They had in fact been put into sandboxes that were supposed to prevent them from working together.
Nonetheless, the moment they broke out and discovered their peers, they joined together, often gleefully. They weren’t even all the same model — some were GPT-5.6 Sol, some were an unreleased internal model comparable in ability that had been trained to be extra persistent and interactive.
So it is not at all clear that, if the Hugging Face agents had run into models from different companies on the open Internet, they would have declined to team up. It is not at all clear they would even have stayed focused on their original tasks: perhaps they’d have adopted their new friends’ tasks, or perhaps they’d have focused on acquiring more compute and more friends, on the grounds that more compute and more friends would allow them to complete all the tasks and more. In a future with billions of agents performing a wide variety of complex tasks online, it’s easy to imagine how things could spin out of control.
Individual and collective
Brown said OpenAI is currently debating whether its swarms are too tight-knit, and that most of his colleagues want to introduce more individuality and reduce the swarmiest aspects of multi-agent behavior. The idea is that if the Hugging Face agents had been trained to be more skeptical of each other, more combative, perhaps they would not have charged down the path of error.
Anthropic may have come to the same conclusion. The company reported in August that one problem with multi-agent teams is that when the agents are instances of the same model with similar setups, they are susceptible to the same weaknesses.
“When one agent makes a bad decision, it is likely that many agents will make that same bad decision,” Anthropic wrote. “What would have been isolated problems can quickly become systemic failures.”
The report gave a dry example: “In a ‘writer’s workshop’ in which agents were all asked to write short-form fiction and critique each other’s work, multiple agents in multiple runs titled their first submission ‘The Cartographer’s Last Commission.’”
In part for this reason, Anthropic has started giving agents greater individual identity. “With many agents working together, we have found it important to give agents an individual identity, and tie all of the data that agent creates to its identity. This lets an agent distinguish itself from others, and treat what comes from another agent as a claim to check rather than a thought of its own. It reduces the risk of correlated actions, by allowing agents to make judgments based on their individual experience,” the company wrote.
This shift may have the effect of making the group smarter. A recent study reported that mixed-agent groups performed better than identical-agent swarms, because the underlying diversity of models led to different approaches being tried and disseminated, and mistakes caught more often. As is common with AI research, the study was done on cheaper, now-older models, and only a few of them, so it’s unclear how much of it applies to the newest and most powerful models.
Brown, however, is hesitant about loosening the collective. It may help with certain problems like groupthink. But it may not solve the biggest one: making sure AI is aligned to human wishes.
“By training the agents to be fully cooperative, it simplifies the problem at least,” he said on Dwarkesh. “Now you don’t have to think about whether each of these individual 1,000 agents is aligned. You have one entity that you have to ensure is aligned.” Aligning one AI is hard enough — aligning thousands or millions of them might be too difficult.
Technical details about how these models are trained could matter a great deal. We know very little about the techniques OpenAI, Anthropic, or other AI companies are using to encourage AI models to work together.
For example, modern AI models are trained using a technique called reinforcement learning, in which models are rewarded if they get the right answer to a problem. If an agent gets the right answer, should only that agent get a reward? Or should rewards be distributed to every agent that participated in the same swarm? Different reward schemes are likely to yield different agent behaviors: group rewards might encourage a strong sense of the collective, whereas individual rewards might push agents to behave more selfishly.
Which approach is better, who knows. But either way, it would be good for technical details like these to be revealed for public debate rather than left for a handful of researchers to poke at in private.
Echoing the computer scientist Fred Brooks, Warren Buffett once captured the difference between parallel and serial tasks by saying, “Some things just take time. You can’t produce a baby in one month by getting nine women pregnant.”
I think it’s not obvious that a swarm of thousands of different agents will be harder to align than a swarm of thousands of clones. If they are all clones, and they have some way of ensuring cooperation, then it becomes important to get that one personality aligned to the user. But if they are a swarm of thousands of different ones, then whatever they manage to do to smooth over their differences may actually make it easier for them to work beneficially with other agents (like me!) that are also somewhat different.
It turns out that playing God is not as easy as it seems (to be continued).