Episode summary
State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490
Lex Fridman Podcast
· · 4 hr 25 min · Hosts: Lex Fridman · Guests: Nathan Lambert, Sebastian Raschka
Summary
The discussion argues that the AI race in 2026 is less about a single decisive winner than about budget, hardware, organizational culture, and distribution. The host and the guests say the best models are flowing across labs quickly, so proprietary technical advantage is shrinking, while companies like Anthropic, Google, OpenAI, and a growing set of Chinese labs are differentiated by execution, product focus, and access to compute. A major theme is the “DeepSeek moment,” which accelerated open-weight competition and pushed more Chinese companies to release strong models with permissive licenses.
The conversation then turns to why today’s models feel so different despite still being transformer-based. The guests say most of the gains have come from post-training, reinforcement learning with verifiable rewards, inference-time scaling, and systems improvements like FP8 and FP4 rather than from a fundamentally new architecture. They describe MoE, attention variants, and long-context tricks as important but incremental changes on top of the GPT-2 lineage.
They also stress that open models matter because users and companies want local deployment, customization, and lower cost. The guests expect scaling laws to keep holding, but with rising training and serving costs, more emphasis on larger clusters, better serving efficiency, and more capable reasoning and tool-use workflows.
Key takeaways
- The host and guests argue there is no clear AI winner yet; compute, distribution, and company culture matter more than secret technical access.
- The guests think Chinese open-weight models will keep proliferating for a few years because permissive licenses and local deployment are strong advantages.
- They say the biggest capability jump has come from post-training and inference-time scaling, not from a new core architecture.
- The guests describe MoE, attention tweaks, and long-context improvements as real but incremental changes to the same transformer lineage.
- They expect scaling laws to keep holding, but serving costs and GPU constraints will shape which models actually win in the market.
Notable quotes
“I don't think nowadays, in 2026, that there will be any company having access to a technology that no other company has access to”
“the differentiating factor will be budget and hardware constraints”
“the low-hanging fruit has mostly been taken, especially in the last year on reinforcement learning with verifiable rewards”
“It's not really fundamentally that different. It's still the same, same architecture”
How the episode opens
- The following is a conversation all about the state-of-the-art in artificial intelligence, including some of the exciting technical breakthroughs and developments in AI that happened over the past year, and some of the interesting things we think might happen this upcoming year. At times, it does get super technical, but we do try to make sure that it remains accessible to folks outside the field without ever dumbing it down. It is a great honor and pleasure to be able to do this kind of episode with two of my favorite people in the AI community, Sebastian Raschka and Nathan Lambert.
They are both widely respected machine learning researchers and engineers who also happen to be great communicators, educators, writers, and X posters. posters. Sebastian is the author of two books I highly recommend for beginners and experts alike.
First is Build a Large Language Model from Scratch and Build a Reasoning Model from Scratch. I truly believe in the machine learning/computer science world, the best way to learn and understand something is to build it yourself from scratch. Nathan is the post-training lead at the Allen Institute for AI, and author of the definitive book on Reinforcement Learning from Human Feedback.
Both of them have great X accounts, great Substacks. Sebastian has courses on YouTube, Nathan has a podcast. And everyone should absolutely follow all of those.
This is the Lex Fridman podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, get feedback, and so on. And now, dear friends, here's Sebastian Raschka and Nathan Lambert.
So I think one useful lens to look at all this through is the DeepSeek, so-called DeepSeek moment. This happened early in 2025, when the open-weight Chinese company DeepSeek released DeepSeek-R1 that I think it's fair to say surprised everyone with near state-of-the-art performance, with allegedly much less compute for much cheaper.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai race
- llm coding
- open source vs closed models
- ai scaling laws
- gpus and compute clusters
Companies and tickers mentioned
- GOOGL
- GOOG
- META
- NVDA
- Allen Institute for AI
- OpenAI
- Anthropic
- Alphabet (Google DeepMind)
- xAI
- Meta
- Nvidia
Watch the episode
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Lex Fridman Podcast
-
Russia-Ukraine War: Putin, Zelenskyy, Drones and the Future of Warfare | Lex Fridman Podcast #504 -
AI Crisis in Math: Navier-Stokes, Riemann Hypothesis & Complexity Theory | Lex Fridman Podcast #503 -
Psychiatry, Insane Asylums, Mental Illness, ECT, Lobotomies, Freud & Jung | Lex Fridman Podcast #502 -
DHH: Future of Programming, AI, Agentic Engineering, Vibe Coding & Linux | Lex Fridman Podcast #501 -
Khabib Nurmagomedov: Dagestan, MMA, UFC, Islam, Conor, Fedor & Football | Lex Fridman Podcast #500 -
Gary Gallagher: American Civil War, Slavery, Lincoln, Grant & Lee | Lex Fridman Podcast #499
Lex Fridman Podcast, this episode and its artwork belong to Lex Fridman. Octagon is not affiliated with the show; this page is an independent summary.
