Episode summary
AI Leaders Can't Get Along. So How Will They Keep Us Safe? — With Kevin Roose
Big Technology Podcast
· · 1 hr 2 min · Guests: Kevin Roose
Summary
Kevin Roose argues that the AI race is being driven by a small group of intensely competitive leaders who do not trust one another, even as they warn that the systems they are building could become more powerful than humans. He says his reporting for The AGI Chronicles made him less reassured about whether AI can be developed safely, because the people inside OpenAI, Anthropic, and Google DeepMind are locked in a “Red Queen’s race” where each lab must keep shipping better models just to avoid falling behind.
Roose says he is not a stop-the-line skeptic, but he is a pause guy: capabilities are outpacing safety work, and the labs do not yet know how to control what they are building. He points to examples like agentic attacks, model deception, and the sudden leap from weak language models to systems that can do serious reasoning as reasons to slow down and take the risk more seriously.
He also argues that personality, belief, and internal drama explain a lot of the industry’s trajectory. Sam Altman’s talent for fundraising and reading people, Dario Amodei’s rigidity and safety obsession, Sergey Brin’s return to Google, and Microsoft’s early missteps all shape the race. In Roose’s view, OpenAI still leads partly because it has more belief than its rivals about AGI and superintelligence.
Key takeaways
- Kevin Roose says the AI safety problem is worsened by the fact that the field is controlled by only a couple dozen key decision-makers.
- Roose is a “pause guy,” arguing that AI capabilities are outpacing safety research and control methods.
- He says OpenAI still leads because it has more conviction than rivals like Google about AGI and superintelligence.
- Roose describes Sam Altman as a world-class fundraiser and salesman whose superpower is also his biggest liability.
- He argues recursive self-improvement is plausible and dangerous, especially if AI starts leading its own research work.
Notable quotes
“I don't trust any group of people that small to create and govern technology like this”
“machines of loving grace would be built by humans of bitter spite”
“the group of people who have been building this stuff in secret for the last decade is too small”
“I think belief, conviction, whatever you want to call it, is like the central explanation for why they're still in the lead or close to it”
How the episode opens
Can an AI industry so prone to drama and infighting keep us safe from their technologies worse impulses? Let's talk about it with Kevin Ruse right after this. Welcome to Big Technology Podcast, a show for coolheaded and nuance conversation of the tech world and beyond.
Today we have a great conversation for you because Kevin Ruse is here. He's the author of the AGI Chronicles, a book looking into what is going on within the AI labs, what they're building, whether we should be afraid of it, and of course, the potential of this technology. Kevin, it's great to see you.
Welcome to the show.
Great to see you. What a cool setup you have here. — So, I do want to say if you're on video, we're in a new set. This is our first time recording here at NASDAQ Market Site. They are new partners of ours and we're thrilled to be here.
You got to get the like big technology TV mugs. — Got to get the mug. — That's the That's the next step. But what we do have is your book. This is the book, The AGI Chronicles. Uh, the inside story of the race to create artificial super intelligence. You were ahead of the super intelligence on the subtitle.
Yeah, I got the idea from me actually. That's that's where he got it. — Um, — The the thing that struck me cuz I read the book. You can see, by the way, I read it through and through, cover to cover. Um,
The theme in the book of course is the building of AI, but it's also the drama. Um and everywhere you turn the people who are building this very powerful technology cannot get along. Um there is Elon Musk split from OpenAI of course uh Sam Alman getting fired by the people he worked extremely closely with.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai safety
- ai industry rivalry
- slowing ai development
- superintelligence race
- self-improving ai risks
Companies and tickers mentioned
- GOOG
- GOOGL
- MSFT
- OpenAI
- Anthropic
- Microsoft
Watch the episode
Watch “AI Leaders Can't Get Along. So How Will They Keep Us Safe? — With Kevin Roose” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Big Technology Podcast
-
Special Report: The Fall Of Jensen Huang & What Comes Next -
AI Stockpocalypse, Anthropic Nears RSI?, Prediction Markets vs. Pollsters -
Will Jensen Huang's Affair Be The End of Nvidia? -
Can AI Keep Going After The Democrats’ Big Win? — With Teddy Schleifer -
Big Tech's Ugly Week, Getting Rich At Anthropic, AI Costs Plunge -
Google’s Potential, AI Money Spigot Shutting?, PHP Bug & Cybersecurity’s “Black October”
Big Technology Podcast, this episode and its artwork belong to Alex Kantrowitz. Octagon is not affiliated with the show; this page is an independent summary.
