Episode summary
Ex-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper
Invest Like the Best
· · 1 hr 24 min · Hosts: Patrick O'Shaughnessy · Guests: Neil Movva
Summary
Neil Movva argues that AI is entering a shift from chatbots to background agents that work for hours or days, and that the winning business will be the one that makes those tokens as cheap as possible. Sail Research, he says, is built as a “token factory” with an API for cheap inference and long-running sandboxes for agents, because abundance of intelligence will create new product categories once it becomes 10x cheaper.
He thinks the market has overfocused on low-latency chat and underappreciated long-horizon work such as deep research, cybersecurity, and proactive personal assistants. In his view, the best latency is no latency: the machine should do work in the background before the user asks. He expects background workloads to dominate real-time interaction over time, and he believes the key constraint will be the token budget, not demand.
Movva also gives a hardware-and-systems thesis rooted in his Nvidia background. He explains why GPUs are best at throughput, why batching and interconnects create a latency-throughput trade-off, and why different chips have different comparative advantages. He sees a hybrid future: GPUs for capacity and heterogeneous accelerators for specialized memory or speed, while power, data centers, and cheap supply-side inputs determine who can make intelligence abundant.
Key takeaways
- Neil Movva says Sail Research is optimizing for the lowest cost per token and long-running agent workloads, not chat-style latency.
- He expects background agents to become the dominant use case, with the market shifting toward proactive work done before the user asks.
- Movva argues that deep research and cybersecurity are early winners because they reward large-scale, verifiable token consumption.
- He believes cheaper intelligence will unlock proactive personal agents that understand a user's emails, messages, and likely next actions.
- Movva says Nvidia remains excellent for both low-latency and high-throughput work, but different chips can still win on price or specialization.
- He views the real opportunity as arbitrage across software, chips, data centers, and power rather than betting on one perfect chip.
Notable quotes
“My job is to make the tokens as cheap as humanly possible. I will achieve that and I will do it through every layer in the stack available to me”
“Sal research is a token factory. We have an API where anyone can send us requests where they can use large language models”
“the future of agentic inference is long horizon tasks. You're going to run the machine for hours or days at a time”
“the best latency is no latency at all. When you wake up in the morning, the work's already been done overnight”
How the episode opens
My job is to make the tokens as cheap as humanly possible. I will achieve that and I will do it through every layer in the stack available to me. I love the supply side levers.
I will use every chip. I'll use every source of power and I will use every piece of land in the United States that's, you know, suitable for this. We still treat the agent as a person that is expensive to consult and you should ask them when you have a hard question.
That's not the way to think about intelligence. It's incredible that the machine can think and we should try to get that into as many hands as as many people as possible.
I think it's important early in these conversations to just say the thing like literally what you're building and what it does today. So maybe just orient us there with a with a brief description like literally what the system is that you're building and why it should exist.
Sal research is a token factory. We have an API where anyone can send us requests where they can use large language models, open source large language models for any task they want. Um, we will serve those tokens to them at a price that is unbeatable in the market.
We also support their ability to build agents on top of this. We host what we call sandboxes, which are longunning agent virtual machines hosted in the cloud that are designed for agents that run for hours, days, or weeks. And so you should think about you as a peer company to others that serve different kinds of inference.
You're serving one specific kind of inference and your goal is to be the absolute cheapest provider and enabler of a certain kind of use of intelligence.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai background agents
- token factory strategy
- gpu stack and chips
- ai data center economics
- open vs closed ai models
Companies and tickers mentioned
- NVDA
- Nvidia
- Sail Research
Watch the episode
Watch “Ex-NVIDIA Engineer: Why AI Is About to Get 1000x Cheaper” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Invest Like the Best
Invest Like the Best, this episode and its artwork belong to Colossus | Investing & Business Podcasts. Octagon is not affiliated with the show; this page is an independent summary.
