Episode summary
World's Top Researcher on AI, LLMs, and Robot Intelligence
Invest Like the Best
· · 1 hr 15 min · Hosts: Patrick O'Shaughnessy · Guests: Sergey Levine
Summary
Sergey Levine argues that the core robotics problem is not building a single specialized machine, but a robotic foundation model that can control many embodied systems across many tasks and environments. He frames Physical Intelligence as pursuing “physical intelligence” in the same way LLMs generalize across language: by learning from broad data, then adapting to new applications rather than hard-coding one robot for dishes or laundry.
A central theme is that generalization matters more than flashy demos. Levine says the hardest part is not making a robot succeed in one pristine setting, but making it robust in open-world conditions, including long-tail scenarios where common sense is needed. He says multimodal LLMs are now important because they can supply that missing prior knowledge, while chain-of-thought-style reasoning and reinforcement learning help robots plan, improve, and become faster over time.
Levine also emphasizes that the field may become more like a toolkit than a single robot form factor. He is bullish on humanoids as a compelling demo, but argues the real opportunity is a model that can adapt to arms, hands, bulldozers, or other embodiments. He thinks the biggest near-term bottlenecks are data, deployment constraints, and the social acceptance of imperfect robots in homes and care settings.
Key takeaways
- Levine says Physical Intelligence is building robotic foundation models that can control “basically any embodied system” and generalize across tasks, robots, and environments.
- He believes the key advantage of foundation models is not just task performance, but the ability to improve from broad data and new experience without hand-engineering every system.
- Levine says multimodal LLMs matter because they can inject common sense into robotics, especially for long-tail situations that are hard to cover with teleoperation data alone.
- He thinks humanoids are useful for demonstrations, but the bigger prize is a general model that can adapt to many form factors, from robot arms to very different embodiments.
- Levine says the hardest practical barrier to home robots may be social and safety acceptance, not just raw technical capability, especially for tasks involving children or fragile objects.
Notable quotes
“Fundamentally, the goal of physical intelligence is to develop robotic foundation models that can control basically any embodied system to do any task”
“the key is this generality particularly with respect to improvement”
“We need lots of data. The cool thing about being able to build robots is that ultimately they don't have to be constrained to look like humans at all”
“The same, by the way, also applied to getting systems to work on different embodiment where we could get our models to work on all sorts of other robots”
“I think we should handle it in a general way because otherwise it's just really hard to like to get a handle on this”
How the episode opens
My guest today is Sergey Lavine, one of the co-founders and researchers at physical intelligence. As a disclaimer, I'm an investor in physical intelligence because I believe it's one of the most important companies tackling the problem of robotics. As you hear us discuss today, robotics has what I would call a scarecrow problem.
All of these amazing physical devices are becoming ever more possible in all sorts of cool permutations, but what they all really need is an intelligence, a brain, and that is what they're developing at physical intelligence. are trying to develop foundation models that can make any physical robot do any task in any environment. That challenge is daunting and has required many of the world's best researchers, Sergey as one of those leaders, coming together to try to solve this problem.
The nature of our conversation today is all of the problems facing robotics and all of the promise of solving these problems across the world. I hope you enjoy this great conversation with Sergey Lavine. This is going to be a real treat and a blast to learn about possibly the most exciting impactful area of technology being developed.
Just to set the stage before we go back in time, maybe you could just define physical intelligence as you see it.
Fundamentally, the goal of physical intelligence is to develop robotic foundation models that can control basically any embodied system to do any task. But broadly speaking, you could imagine that in the same way that a language model is kind of rapidly evolving towards a system that can do any task that can be expressed in language, what we would like is to build a new class of models that can do any task that can be done by a physical actuated device.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- physical intelligence
- general purpose robotics
- llm vs robotic learning
- robotics benchmarks
- robotics data and simulation
Companies and tickers mentioned
- Physical Intelligence
Watch the episode
Watch “World's Top Researcher on AI, LLMs, and Robot Intelligence” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Invest Like the Best
Invest Like the Best, this episode and its artwork belong to Colossus | Investing & Business Podcasts. Octagon is not affiliated with the show; this page is an independent summary.
