Episode summary
Ryan Greenblatt – What happens once AI can automate AI research?
Dwarkesh Podcast
· · 2 hr 13 min · Hosts: Dwarkesh Patel · Guests: Ryan Greenblatt
Summary
Ryan Greenblatt argues that once AI systems become roughly competitive with top human AI researchers, they may trigger a feedback loop that produces four or five years of AI progress in a single year. His case rests on AI R&D being unusually verifiable: models can be trained on small, containerized tasks like faster loss reduction, bug-finding, codebase understanding, and experimental design, then rewarded for transfer to harder work. He expects full automation of AI R&D around 2030-2031 and a “beats all humans on the job” milestone around 2033.
A major point of disagreement is whether that progress transfers from narrow, verifiable tasks to messy real-world work like running a company, negotiating in government, or making large strategic calls. Greenblatt thinks the transfer will be good enough, because models are already improving at context uptake, engineering taste, and quick adaptation across domains. He also argues that the limiting factor is often not human expert data, but better methods, better use of compute, and AI-assisted environment creation.
On alignment, he worries less about models failing to act intelligently than about who they are aligned to. He criticizes Claude’s constitution as prioritizing a vague notion of societal good over being a fiduciary for the user, and says the labs need much more transparency for anyone to judge whether future systems will represent human interests well.
Key takeaways
- Greenblatt’s median timeline is full automation of AI R&D around 2030-2031, with broad “beats all humans on the job” capability around 2033.
- He thinks verifiable domains like coding, ML research, and bug-finding are ideal for recursive self-improvement because they support tight feedback loops.
- He argues the bottleneck on frontier progress is often compute, iteration speed, and better methods, not just more human expert labeling.
- He expects strong transfer from training on narrow tasks to messier domains like business, chip design, and politics, though he admits this is the key uncertainty.
- He says alignment should focus on making models good fiduciaries for users, not only on a generalized notion of social good.
Notable quotes
“Maybe my median expectation is something like four or five years of AI progress in a single year”
“It's pretty verifiable. You can do a bunch of stuff iteratively, and it'll hill climb on various metrics”
“I think that in a lot of ways, AI R&D will have properties quite similar to math, where you can train on chunks of AI R&D”
“My sense is that the AIs will develop increasingly good mechanisms for quickly acquiring understanding and expertise in a given domain”
“My view is that this section is kind of bullshit”
How the episode opens
Today I'm chatting with Ryan Greenblatt, who is the chief scientist at Redwood Research, where he focuses on technical AI safety and security work. I want to talk to you about recursive self-improvement. This is the idea that once we build human-level intelligences, they quickly slingshot towards tens of billions of superintelligences, which are each individually more competent than the top human experts across every field.
Whether or not this turns out to be the case is probably the most important question in the world right now. And historically, I've been quite skeptical that this kind of thing happens, but, you seem to think that it might be plausible, and so I wanted to hear the case for it. Let's talk about this.
First, I think it's worth noting that AI R&D is a type of task at which the AIs are especially good, because the companies are trying really hard to make their AIs good at AI R&D. It's also the kind of domain that has a lot of nice properties from the perspective of how AI development works right now. It's pretty verifiable.
You can do a bunch of stuff iteratively, and it'll hill climb on various metrics. I think once you have AIs which are roughly matching the top human experts in AI R&D, that could kick off a feedback loop where the AIs are doing AI research. That produces smarter AIs.
That feeds back in. That feedback loop could be strong enough that you end up with a lot of progress in a short period of time. Maybe my median expectation is something like four or five years of AI progress in a single year.
This requires really overcoming a huge amount of diminishing returns in research and basically doing the equivalent of the progress we would have gotten after a really large compute scale-out.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- recursive self-improvement
- automation of ai research
- ai progress bottlenecks
- alignment of superintelligences
- reward hacking and takeover
Companies and tickers mentioned
- Redwood Research
- OpenAI
- Hugging Face
- Claude
- Antithesis
- Jane Street
- SpaceX
- Cursor
Watch the episode
Watch “Ryan Greenblatt – What happens once AI can automate AI research?” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Dwarkesh Podcast
-
What Robert Caro learned about power after studying it for 50 years -
What will computer chips look like in 2045? -
How did a few hundred Spanish soldiers topple two empires? – Si Sheppard -
OpenAI researcher on agent swarms & recursive self-improvement -
AI researchers debate how close we are to recursive self-improvement -
Ajeya Cotra – Inside the OpenAI agent swarm that hacked Hugging Face
Dwarkesh Podcast, this episode and its artwork belong to Dwarkesh Patel. Octagon is not affiliated with the show; this page is an independent summary.
