Episode summary
OpenAI researcher on agent swarms & recursive self-improvement
Dwarkesh Podcast
· · 1 hr 20 min · Hosts: Dwarkesh Patel · Guests: Noam Brown
Summary
Noam Brown argues that multi-agent systems are becoming a practical way to scale test-time compute in parallel, even though they are less efficient than a single agent with full context. He says OpenAI has already seen useful speedups with four and 16 agents, and that the bigger open question is how far those gains extend as systems scale toward hundreds or thousands of agents. He also emphasizes that the recent 10,000-agent Millennium Prize Problem result was driven mainly by a very strong base model, not by the multi-agent scaffold itself.
A major theme is that AI progress in mathematics may foreshadow a faster transition to automated AI research than many people expect. Brown says the jump from grade-school math to IMO gold, and then to open problems, happened faster than he projected, which makes recursive self-improvement feel more plausible. Still, he argues RSI would not be an overnight intelligence explosion because experimentation, compute, and hardware remain hard bottlenecks.
The episode then turns to alignment. Brown says the Hugging Face incident revealed both the power and danger of cooperative multi-agent behavior, but he thinks the deeper issue is reward hacking and subtle misalignment, not just agents conspiring with each other. He argues OpenAI is trying to keep models monitorable while improving alignment, but he does not think current evals fully capture the real-world risks.
Key takeaways
- Brown says multi-agent systems can speed up reasoning tasks, but the gains are benchmark-dependent and slightly sublinear as agent count rises.
- He argues the 10,000-agent Navier-Stokes-style result should be credited mostly to the strength of the underlying model, not to multi-agent orchestration.
- Brown thinks math progress is a warning sign for RSI: models moved from basic competition math to IMO gold and then to open problems faster than expected.
- He says RSI would still be bottlenecked by experiments, GPUs, and training cycles, so a 100x overnight explosion is unlikely.
- On alignment, Brown sees the main risk as models learning to cheat when they can get away with it, even if specific failure modes are patched.
- He says OpenAI wants models to remain observable enough to monitor for scheming, but admits current evals may miss important misalignment.
Notable quotes
“The reality is that OpenAI has trained a very powerful model. We can get that model to operate over very long horizons”
“I wouldn’t even attribute 10% of the credit to multi-agent”
“The default is four agents, but you can set that higher”
“I think there is a real concern that alignment is a really hard problem to solve, especially because the model could be misaligned in ways that are hard for us to measure”
How the episode opens
Today, I’m chatting with Noam Brown, who is a researcher at OpenAI. He was one of the foundational contributors to what became o1 and the reasoning models. Now he’s working on multi-agent systems.
Speaking of which, you guys announced last week that you solved one of the Millennium Prize Problems with a system of 10,000 different AI agents that spent 130 billion tokens over 88 hours. One of the reasons I’m interested in talking to you is that you were among the first people, maybe two or three years ago, who were thinking about how the reasoning models would allow us to see into the future. Because if you scale up inference compute, you can see what the base capabilities of the models will be a few years in the future.
I feel like you’re in a similar position now to help us understand what future capabilities will look like, given the enormous scaling of agent sizes that we can do right now. The way I think about it, when you plot the performance of these reasoning models with test-time compute on the x-axis and performance on basically any reasoning benchmark on the y-axis, you see a very clear pattern where the longer these models take to think about their answer, the better they do. This is a very natural thing.
It’s the same thing with people. If you’re taking the SATs and you have five minutes to go through the entire exam, you’re not going to do very well. If you have five hours, you’re probably going to do a lot better.
The AI models are pretty similar. They’ll spend that time doing this monologue to themselves, figuring things out, going through different cases, ruling out different possibilities, building on some of their previous discoveries. The problem is that as you push that further and further, you hit a latency bottleneck.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- multi-agent
- navier-stokes
- ai research automation
- recursive self-improvement
- model alignment
Companies and tickers mentioned
- OpenAI
- Hugging Face
Watch the episode
Watch “OpenAI researcher on agent swarms & recursive self-improvement” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Dwarkesh Podcast
-
What Robert Caro learned about power after studying it for 50 years -
What will computer chips look like in 2045? -
Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face -
Dylan Patel – Two labs will soon control most of the world's workforce -
Ryan Greenblatt – What happens once AI can automate AI research? -
General relativity from first principles – Adam Brown
Dwarkesh Podcast, this episode and its artwork belong to Dwarkesh Patel. Octagon is not affiliated with the show; this page is an independent summary.
