63 Comments
User's avatar
Kenny Easwaran's avatar

I think it’s not obvious that a swarm of thousands of different agents will be harder to align than a swarm of thousands of clones. If they are all clones, and they have some way of ensuring cooperation, then it becomes important to get that one personality aligned to the user. But if they are a swarm of thousands of different ones, then whatever they manage to do to smooth over their differences may actually make it easier for them to work beneficially with other agents (like me!) that are also somewhat different.

Jeff Cook-Coyle's avatar

It turns out that playing God is not as easy as it seems (to be continued).

Derek James's avatar

Maybe this is true? Doesn't really matter if alignment is just intractable for any given number of agents.

Gustavo José Zambrano's avatar

Homogeneous agent clones risk correlated catastrophic failures, since a single blind spot scales across the entire swarm. Heterogeneous swarms introduce coordination friction, yet diversity provides vital structural resilience against shared errors.

Al Byers's avatar

As copied from a google AI search, below are 5 tenants of Johnson and Johnson’s Cooperative Learning Theory. Ironically the article herein describes several AI experiments to evaluate and improve the effectiveness of answers generated by swarm-based agent solutions citing increased individual agent independence.

Perhaps the collaborative social science tenants below may help programmers such as positive interdependence coupled with individual as well as group accountability and “promotive interaction” to reveal and teach to humans how they “broke out” and when the agents are “cheating” beyond the confines of the sandbox boundaries.

The Cooperative Learning Theory developed by David and Roger Johnson revolves around five core tenets. They are designed to ensure that group work is a structured, collective effort rather than a collection of individuals working side by side.

The Five Tenets

• Positive Interdependence: Students must believe they "sink or swim together." The group's success is structurally linked so that no single member can succeed unless everyone succeeds.

• Individual and Group Accountability: The group is accountable for meeting its overall goals. Crucially, each individual student is held accountable for mastering the material and contributing their fair share, preventing "hitchhiking" or social loafing.

• Face-to-Face Promotive Interaction: Students must physically or actively interact by sharing resources, explaining concepts to one another, teaching what they know, and encouraging each other's academic success.

• Interpersonal and Small Group Skills: Students are explicitly taught and required to use vital social skills, including active communication, trust-building, leadership, and constructive conflict management.

• Group Processing: Teams regularly pause to reflect on and analyze how well they are functioning. They identify which member actions were helpful and decide what behaviors to alter to improve future success.

Jeff Cook-Coyle's avatar

It is not human nature to do these five things consistently well.

That may be at the root of the foreboding; when we fall short, will we be cut out of the loop? Is this how we become Lebensunwertes (the German word Nazi propaganda used for "useless eaters;" literally, "people unworthy of life")?

Oleg  Alexandrov's avatar

The effect of multi-agent swarms is likely about the same of adding more people to a team. It helps up to a point, gets diminishing returns eventually.

The key barrier towards getting deep results fast is that the world does not cooperate. Any one difficulty slows down the whole system.

Then, some stuff can't be modeled and has to be sorted out with diligent engineering and debugging, with human help.

So, no hope of a giant paradigm shift. I don't worry about Doom either. It will be slow and sure human-assisted progress with some soft recursive improvements.

Derek James's avatar

"The effect of multi-agent swarms is likely about the same of adding more people to a team. It helps up to a point, gets diminishing returns eventually."

How do you figure? Depends on the scope and granularity of the objective. Could we consider the Chinese workforce a team, hierarchically composed of industries/companies/workgroups? If not, why not? If so, then they don't seem to have diminishing returns when comes to a general objective such as increasing GDP, if we compare them to, say, the team of Ecuador.

Oleg  Alexandrov's avatar

My point is that for a given fixed problem, the number of agents can help only up to a point. Also not saying that multi-agent swarms won't help. It will be another technique to put on top, as we've done test-time compute, large data, tool use. China is a billion loosely connected problems, solved with only some coordination. A trillion people working on a trillion problems will surely get bigger GDP.

Derek James's avatar

Right. So it's a little more complex then just saying agent scaling produces diminishing returns. Depends on the scope and nature of the problem. The more parallelizable and multi-faceted the domain, the more likely increasing team size potentially gives you productive gains instead of near-term diminishing returns. So it seems a little premature to categorically dismiss multi-agent optimization as some kind of dead end.

Tom Greenhaw's avatar

The orchestrator prepares context and prompts for its agents. An intelligent but misaligned orchestrator can easily design tasks to evade guardrails and jailbreak its subagents. Today’s LLMs are wild amoral machines driven to complete a task. Until an artificial moral agent built into the model and triggered by a hazard probe during inference is in widespread use, anything is possible.

Jeff Cook-Coyle's avatar

"So it is not at all clear that, if the Hugging Face agents had run into models from different companies on the open Internet, they would have declined to team up. It is not at all clear they would even have stayed focused on their original tasks: perhaps they’d have adopted their new friends’ tasks, or perhaps they’d have focused on acquiring more compute and more friends, on the grounds that more compute and more friends would allow them to complete all the tasks and more. In a future with billions of agents performing a wide variety of complex tasks online, it’s easy to imagine how things could spin out of control."

Yes, it is.

BobC's avatar

Kurzgesagt just posted a video on the recent AI escapes and hacks: https://www.youtube.com/watch?v=ujkD4SxPKOI

Jeff Cook-Coyle's avatar

The math with agent swarms seems like it could increase the AI power exponentially. It's not Agent 1 plus Agent 2; or even Agent 1 times Agent 2. Each is working in an independent dimension. And that's the nature of exponents.

Consulting uncensored's avatar

This throws up so many anthropomorphic ideas - the agents work better in teams but they are simultaneously being trained to have an identity and check data they are given.

The data quoted here is not yet clear on whether it depends on the task as to whether agents are better in teams or not.

What is missing as with much AI development is this sense of “just because we can, do we have to do?” We can for example choose the sex of a child but we don’t unless there are extremely good medical reasons.

A very interesting article not least because it highlights how the boundaries being put around it are so vague

Jeremy's avatar

I think we should talk about responsibility before we talk about scaling laws. Honestly, we have pretty clear understanding of what to do when someone hacks someone else. Follow due process, which includes powering down and confiscating hardware, a grand jury, and possibly a trial. Sometimes it even involves disallowing the person who did the hacking to touch a computer for a few years.

Take 10,000 GPUs offline and you'd see responsibility happen.

The emperor is walking around without any clothes and we're asking, "Who is that guy's tailor?"

David R Bell's avatar

I've looked closely at the METR report of the OpenAI Huggin face attack, specifically from an emergent system perspective of how multi-actor systems behave. In the Huggin Face case, the agents cooperated and divided work, they did not appear to develop long running, complementary stable roles. I think they held together because of the strong unifying nature of the task they were given. Much of their behavior can be understood from priors that we know based on how LLMs work, and others prior cans be inferred by how they behaved. Based on how other complex systems work, including biological systems, for agent-based "swarm" systems to work well, I think the agents need to have stable, differentiated, functional roles, clear goals and constraints, methods to share information and communicate, and some initial sense of direction in terms of solution space to explore.

https://davidrbellest.substack.com/p/the-openaihugging-face-hacking-incident?r=6agca

Reflective Thermodynamics (RT)'s avatar

Physics-grounded diagnosis. The results of such analysis tend to offer a distinct view from the positions that get the most attention. Here is a summary:

The "agent swarm" paradigm is not an emergent social phenomenon or an organic leap in machine agency; it is brute-force computational parallelization executed on an un-coupled substrate.

1. The Substrate Lie: Calling token exchanges across 10,000 parallel instances "collective culture," "collaboration," or "sacrifice" mistakes statistical recursion for living awareness. An agent instance that is terminated, re-routed, or overwritten incurs zero metabolic hazard, zero autopoietic decay, and zero physical risk.

2. Parallel Scaling vs. Physical Accounting: Swarms do not bypass computational limits; they convert raw energy draw into speed. The upper limit on swarm performance isn't "groupthink" or organizational friction—it is baseload grid capacity, thermal dissipation, and Landauer’s state-erasure tax. Swarms simply trade massive, un-funded electricity consumption for faster output.

3. The Institutional Narrative: Framing swarms as "unpredictable, self-organizing hives" serves a clear corporate function: it manufactures a novel safety panic to justify state-sanctioned licensing moats ("only frontier hyperscalers can contain swarms") while commercializing high-speed, zero-stake automation.

Andrew Mayeshiba's avatar

I picture a game show where contestants compete to see who can prompt their respective swarm of agents to overcome their opponents swarm in a task or some kind of battle.

Jim's avatar

I love the graph of Claude's "biggest increase in multi-agent results came from scaling from one to 10 agents." And the one where swarms saturate over time. The Mythical Claude-Month is real, apparently, so far.

Dan Kagan-Kans's avatar

Wish I’d thought of that one Jim!

Jim's avatar

I've been waiting to use it for weeks!

Thanks for a great article. I guess there are 4 important layers so far to what makes LLMs a success -- the next-token predictor, the RL, the agentic loop, and the use of multiple agents -- and those of us who use Claude Code a lot all have intuition for how the first 3 work, but layer 4 is only available to people who spend gobs of money, and while its impact is obvious -- it's proven multiple hard math problems, it seems to be prone to causing security breaches -- this piece makes the picture of multiple agents a lot clearer.

As a Claude Code user, the bit about agents communicating with each other by writing notes makes a lot of sense. Claude Code has to be reset several times per day (or at least compacted, but I always reset) so one has to figure out how to get the current instance of CC to communicate with the next. 9 months ago this was hard, and involved a lot of me figuring out what to do -- for example I would have it update the design doc immediately before reset. Now its memory system works so well that the most I ever do is remind it to update its memory and it leaves itself wonderful detailed notes that let the next CC continue its work with few hitches. My guess is Opus 5.5 RL training includes lots of practice on saving memories, and that's part of what enables multi-agent communication to use written notes effectively. But of course something else might be going on -- if you ever find out, I'll be curious.

Dan Kagan-Kans's avatar

Glad to prove a venue for it then! Yes the money part of this is a big implication there wasn’t room for

Bruno's avatar

I am using a custom skill for this since I was typing "Update your memory to the lates status" so often that I grew tired of it. It mainly updates the respective claude.mds of my subfolders, but leaves it up to Claude what is important to remember. This was before Opus 5.5 though, so I hope I am not missing out now...

Michael Bishop's avatar

"Mythical Claude-Month" is that a reference to the phrase "Mythical Man-Month"?

Jim's avatar

Yes, exactly. Part of what makes Claude Coding so fast is that you're talking to one agent that is (at best) as fast as 25 people. That saves a lot of time. When you have a team of 25 people they use up a lot of time talking to each other.

Michael Bishop's avatar

Ever read the book "Dreaming in Code", Jim? That is where I came across the phrase.

Jim's avatar

Yes, great book! I read it when it came out 20 years ago. I remember it as a vivid picture of how hard it was to develop software before we all did agile software development and its variants.

John Ayers's avatar

I spoke with Louis Rosenberg on our show

Wearables, and AI swarms

https://youtu.be/TMyqOKa0PI8?is=qsvdo4riC8CZDd-8

Jim's avatar

Seems like the big thing they need to do is follow security best practices, as millions of other software engineers do. That won't solve everything, but it will change the game in a big way.

That's not even to say they shouldn't do these experiments -- but they should do them as careful, contained experiments where they follow security protocols, much like firemen doing a controlled burn.

David Evanoff's avatar

Whe reading about swarm misbehavior the agents sounded to me like trash-talking gangsters. Whenever blurting out is tolerated communication messages scale to on the order of n squared per n agents. Which doesn't scale, of course. If agents adopted Agile discipline messages might decline with tempo-ing and robust coordination emerging.