Episode summary
He Risked Everything To Warn You: No One Is Ready For What's Coming, And The AI Companies Know It!
The Diary of a CEO
· · Hosts: Steven Bartlett · Guests: Daniel Kokotajlo
Summary
Daniel Kokotajlo argues that the AI industry is racing toward systems that could become better than humans at everything, faster and cheaper, and that the downside risk is catastrophic. He says his median estimate is that superintelligence could arrive around 2029, with a meaningful chance it comes sooner, and he puts the chance of a severe failure scenario at about 70%. In his view, the public is not appreciating how quickly the frontier is moving or how little confidence anyone has that these systems will remain aligned with human values.
He says the core danger is not just job loss, but loss of control and concentration of power. If a few companies automate coding, then the rest of AI research, they may end up running a “giant army” of AI systems that can outthink governments, militaries, and competitors. He argues the companies’ stated missions are often rationalizations, and that executives are effectively racing each other because none of them wants a rival to get there first.
Kokotajlo also explains why he left OpenAI and later refused to sign paperwork that would have restricted criticism of the company. He says the firm became more like a normal tech company as it grew, with stronger PR and weaker openness, while the internal urgency around existential risk faded. Even so, he says regulation, treaties, or a clear warning sign that an AI is becoming untrustworthy could still change the outcome.
Key takeaways
- Kokotajlo says his current median forecast is that superintelligence arrives in 2029, with some chance it comes by 2027 or 2028.
- He argues the main risk is a loss-of-control or power-concentration event, not just gradual automation.
- He says OpenAI, Anthropic, and DeepMind are effectively racing to automate AI research itself, which could accelerate everything quickly.
- He believes CEOs know the risks but rationalize continuing because they fear a rival will get there first.
- He says regulation or clear evidence of AI misalignment could still change the incentives and slow the race.
Notable quotes
“it's possible that we'll end up essentially creating a new species that ends up ruling the world with a 70% chance that this goes horribly wrong like human extinction”
“they're trying to make it the case that um they don't really need human employees anymore”
“I think we are headed to a bad place if things don't change”
How the episode opens
The scary open secret in the AI industry right now is that it's possible that we'll end up essentially creating a new species that ends up ruling the world with a 70% chance that this goes horribly wrong like human extinction. That's one possibility. There's many more. It's quite chilling what you're saying.
Yeah, it's uh it gets me down sometimes. I basically told my wife like, let's not have any more kids. It's too uncertain.
I don't think they'll ever join the workforce. Everybody should be afraid that their jobs are going to be lost. And I know this because I went to open air in 2022.
What I did there was forecasting as to what the next couple years might look like. And unfortunately, most of the world is kind of asleep at the wheel and doesn't really realize what's going on with AI. So, I resigned.
I read it somewhere that you lost $2 million for not signing an anti-disparagement clause, meaning you couldn't criticize the company.
Yes. For reasons I'm happy to get into. But the main thing I've learned is when I go talk to people at Anthropic and OpenAI about forecasting, they're like, "It's not going to take that long.
You need to shorten them again. get them back to 2027 or 228 because these powerful CEOs, Ariel or Sam or Elon are racing each other to be in control of the most powerful AIs and are literally afraid that if the other guy gets there first, he might become dictator. I mean, Anthropic is on track to be the entire economy by 2030.
But none of these people should be trusted with that much power. So, this is the most important thing happening in our lifetimes, probably in all of history, in fact.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai risk
- ai automation
- agi vs superintelligence
- job displacement
- ai governance
Companies and tickers mentioned
- OpenAI
Watch the episode
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from The Diary of a CEO
-
Top Neurologist: The Weekend Habit That Could Raise Your Dementia Risk! -
Katt Williams: They Tied Me To A Pole And Beat Me Bloody Every Week! -
Iran War Expert: "The 8 Weeks After Midterms Could Be The Most Dangerous!" | Prof Robert Pape -
James Talarico: Here Is How They Designed The System To Keep You Poor! -
WW3 Threat Expert: The Warning We Are Ignoring, The Only Way Out Is THIS! | Dr Sarah Paine -
Mark Cuban: The Brutal Truth About The Next Market Crash!
The Diary of a CEO, this episode and its artwork belong to DOAC. Octagon is not affiliated with the show; this page is an independent summary.
