Episode summary

Jev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida

20VC · · Hosts: Harry Stebbings · Guests: Diogo Almeida

Jev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida – 20VC

Summary

Diogo Almeida argues that the main barrier to AI automation is not raw capability but reliability: models can be brilliant most of the time and catastrophically wrong the rest, which makes them unsuitable for high-stakes background work. He says Jev is built around “reliability of intelligence,” not speed or cost, and that the field has over-optimized for human preference and benchmarks instead of real automation. In his view, the industry’s big lesson is that data matters more than compute, and that the most important question is choosing the right task before worrying about model size or algorithmic tricks.

He is skeptical of synthetic-data hype, open-ended benchmark chasing, and most “neo labs,” which he calls wasteful. He argues that the best data is task-shaped data that pushes the frontier models in the right distribution, not generic enterprise data lakes. He also says open-source and Chinese models are often compared on misleading benchmarks, and that many labs and copycat products are optimizing for optics rather than true intelligence.

On the commercial side, Diogo thinks incumbents like SaaS companies have an advantage because they know workflows and can engineer around them, but he rejects the idea that every company should “own its own intelligence” via bespoke models. He expects decision-model usage to expand dramatically and believes background automation will eventually dwarf token-heavy chatbot and coding-agent use cases, if the field can solve reliability and efficiency.

Key takeaways

  • Diogo Almeida says reliability of intelligence is the real bottleneck, because models that are 95% right and 5% wildly wrong cannot safely automate important work.
  • He argues the field’s “bitterest lesson” is that data matters more than compute, and the right task matters even more than the data itself.
  • Diogo calls benchmarks “total bullshit” and says they create perverse incentives that hide real model quality.
  • He thinks most neo labs are wasteful, while better ROI comes from task-specific engineering and data aimed at the frontier use case.
  • He believes incumbents like SaaS firms have an edge because they understand workflows, but he doubts that most companies should train their own intelligence models.
  • He expects decision-model usage to scale far beyond current LLM token usage if reliability enables real background automation.

Notable quotes

“Almost everyone who's paying attention to AI knows reliability is the absolute largest bottleneck”

— Diogo Almeida

“I would argue that the way to do that um oh man I wish I could like bring up my slides here but I will I'll I'll try to use my hands I guess”

— Diogo Almeida at 4:51

“benchmarks are total [ __ ] All benchmarks are bad”

— Diogo Almeida at 53:16

“I am concerned for the world. I think I think that throwing money into the furnace is a bad thing for the world”

— Diogo Almeida at 17:18

Topics

  • ai reliability
  • training data
  • synthetic data
  • decision models
  • ai market bubbles

Companies and tickers mentioned

  • TypeSafe AI
  • OpenAI
  • Sequoia
  • Andreessen Horowitz
  • Harvey
  • Neo Labs

Watch the episode

Watch “Jev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida” on YouTube

Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.

More from 20VC

All 20VC episodes →

20VC, this episode and its artwork belong to Harry Stebbings. Octagon is not affiliated with the show; this page is an independent summary.