Episode summary
How OpenAI Builds for 800 Million Weekly Users: Model Specialization and Fine-Tuning
The a16z Show
· · Hosts: Martin Casado · Guests: Sherwin Wu
Summary
Sherwin Wu says OpenAI has shifted from the old idea of one model that would do everything to a portfolio of specialized systems, with different models better suited to different tasks and products. He argues that developers form real relationships with model behavior, which helps explain why API customers often stay with a model family instead of swapping providers easily. He also says OpenAI’s role is not just a horizontal API business or just a vertical consumer app business, but both: ChatGPT drives massive first-party reach, while the API extends OpenAI’s intelligence into other products and workflows.
Wu frames fine-tuning and reinforcement fine-tuning as a response to customer data reality. Companies already sit on large proprietary datasets, and OpenAI’s newer RFT approach lets them use that data more effectively than earlier supervised fine-tuning, potentially improving a model to near state-of-the-art on a specific use case. He says OpenAI has even tested incentives such as discounted inference or free training in exchange for sharing data.
He also argues that prompt engineering has given way to context engineering: what matters now is the tools, retrieval, and data you place around the model. On pricing, he says usage-based billing remains the most natural fit for AI APIs, while outcome-based pricing sounds appealing but is harder to operationalize and may converge back toward usage anyway. On agents, he describes them less as a separate category and more as another interface for deploying OpenAI’s core intelligence, alongside the API, ChatGPT, Codex, and Sora.
Key takeaways
- Wu says OpenAI now sees “a bunch of specialized models” as the likely future, not one universal AGI model.
- Wu says API retention is surprisingly high because users and developers build habits, tooling, and workflows around specific models.
- Wu says reinforcement fine-tuning is the main unlock because it lets customers use proprietary data to push performance much further than earlier supervised fine-tuning.
- Wu says prompt engineering matters less than context engineering, meaning retrieval, tools, and data orchestration are now the real levers.
- Wu says usage-based pricing best matches how AI APIs are actually consumed, while outcome-based pricing is harder to measure reliably.
- Wu says agents are not a separate business line so much as another interface for OpenAI’s underlying intelligence.
Notable quotes
“It's becoming increasingly clear that there will be room for a bunch of specialized models”
“The big unlock that has happened recently is with the reinforcement finetuning”
“prompt engineering is just not going to be a thing”
“the name of the game now is is less on like prompt engineering”
“usage based pricing on the API honestly is because it's been like it's closest to how it's actually being used”
How the episode opens
We want chat GBT as a first party app. First party app is a really great way to get 800 million wows or whatever now. Tenth of the globe, right?
Yeah. Yeah. 10% of the globe uses it every week.
Every week. Yeah. Even within OpenAI, the the thinking was that there would be like one model that rose them all.
It's like definitely completely changed. It's becoming increasingly clear that there will be room for a bunch of specialized models. There will likely be a proliferation of other types of model.
Companies just have giant treasure troves of data that they are sitting on. The big unlock that has happened recently is with the reinforcement finetuning. With that setup, we're now letting you actually run RL, which allows you to leverage your data way more.
Sherman, thanks very much for joining. So, we're being joined by Sherman Woo. Um, it'd be great actually if you provided the long form of your background as we get into this, just for those that that may not know you.
I mean, I I view Sherman as one of the the top AI thought leaders, so I'm really looking forward to this. Yeah. Yeah.
Thanks for having me. I'm I'm really excited to be be on the podcast. Um, yeah.
Yeah, a little bit more of my my background. So, uh maybe we can start from present day and go backwards. So, I I currently lead um the engineering team for the for OpenAI's developer platform.
So, the biggest product in there of course is the the API. Um is there more for the developer platform than the API? Um I just kind of assume there was synonymous.
Well, so I I also think about uh other things that we put into our platform side.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- openai platform
- model specialization
- fine-tuning and rft
- agent builder
- usage based pricing
Companies and tickers mentioned
- OpenAI
- Harmonic Labs
- Rockset
Watch the episode
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from The a16z Show
-
How AI Could Bring Halo and Other Classic Games Back to Life -
Mike Volpi on Why AI Infrastructure Will Outlast the Hype Cycle -
Why HubSpot’s Co-Founder Is Betting Against the SaaSpocalypse -
Inside Salesforce’s Acquisition Strategy for the AI Era -
How AI Is Turning Figma’s Finance Team Into Builders -
The New AI Agents That Shop, Plan, and Gossip
The a16z Show, this episode and its artwork belong to Andreessen Horowitz. Octagon is not affiliated with the show; this page is an independent summary.
