Episode summary
Horizontal vs Vertical Agents: Who Wins? | Model Routing: Commodity or Increasing in Value?
20VC
· · 1 hr 3 min · Hosts: Harry Stebbings · Guests: Harrison Chase
Summary
Harrison Chase argues that building useful agents is still hard because LLMs are nondeterministic, stateful systems that fail in cascading ways once they are connected to tools, data, and memory. LangChain’s role, in his view, is to provide the harness and observability layer that makes those systems controllable, measurable, and portable across model providers. He says better models force the company to delete parts of its product over time, but the need for a neutral layer remains because context and memory should not be locked to one model.
He is especially bullish on vertical agents. Chase thinks the most value will accrue to teams that can define what good looks like in a domain, build the right evals, and tune the harness before touching model weights. He says many companies claiming to build their own models are mostly post-training open-weight models, which he sees as valuable but not the first move for a new product.
On routing, Chase says the real debate is whether it belongs in the gateway or the harness, and he believes domain-specific routing belongs in the harness because it has more context. He is skeptical that routing is a commodity, but he thinks open models and post-training will matter most for simpler tasks like classification, extraction, summarization, and guardrails rather than the core agent loop. He also says memory is still a wide-open problem and will become standard across consumer and enterprise products.
Key takeaways
- Harrison Chase says agent reliability depends more on the harness, context, and memory than on the model alone.
- He is “massively bullish on vertical agents” because domain experts can define better evals and control more of the stack.
- Chase thinks most “own model” efforts are really post-training open-weight models, not training from scratch.
- He argues routing belongs in the harness for vertical agents because it needs domain and state awareness.
- He says memory is still unsolved and will likely become a standard feature in both consumer and enterprise agents.
Notable quotes
“The hard part about building agents is uh they don't behave how you want them to behave”
“I think a lot of it is they are post- training open openw weight models”
“I think routing needs to be in the harness”
“I think this memory layer for agents is still massively unsolved”
How the episode opens
I think we spend 30% of our engineering salary on coding tools. It's actually gone down the last 2 months. While tokens, the number of tokens that we've used has actually gone up.
Today we welcome Harrison Chase, the founder of Lang Chain, multi-billion dollar company. It is a leader in the ability to create and monitor your agents. And today we welcomed him to the hot seat to discuss why it is so hard to build agents today.
What no one knows that everyone should know about what is coming with agents. And then my favorite, some really kooky predictions which no one else believes will happen, but he knows will definitely happen.
I really underestimated the demand for inference. Everything just works. This market is so big. There's just so much demand for it. — Ready to go. Harrison. Dude, I'm so looking forward to this. I've been looking forward to it for a while cuz I remember Miles telling me about you when he was at Benchmark and just singing your praises. So, wanted to make it happen for a while. So, thank you for joining me.
I'm excited to be here. — Now, for those that don't know, 60 seconds, what is Lang Chain and where do you like sit in the stack?
We build tools to make it easy to build agents. So, we build developer tools primarily. We have an open source framework which basically connects the model to everything else that you need to build a useful agent.
So tools, data, computation, things like that. And then we also have Langmith. Langmith is observability and evals.
Agents go wrong all the time. How do you know how they go wrong? And how do you measure them so that you can increase their performance over time?
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai agents
- model routing
- agent evals
- agent memory
- inference market
Companies and tickers mentioned
- LangChain
- Sequoia Capital
- Benchmark
- IVP
- Anthropic
- OpenAI
Watch the episode
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from 20VC
-
Sequoia Leaves London, Close Scout Program & End Evergreen Fund | NASDAQ Loses $2.5T in 24 hours -
Mistral CEO: Why OpenAI & Anthropic Will Come for Your Business | Are Chinese Models Truly Safe? -
Inside WashU's Endowment Fund | SpaceX Made Up 20% of Out AUM | Scott Wilson -
Google Loses $500B in 12 Hours | DoorDash Buys Instacart for $17B | Kalshi Raises $1B -
Modal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute -
David Tisch: Why No Manager Wants to Stay Small & Why Competing with Mega Platforms is a Losing Game
20VC, this episode and its artwork belong to Harry Stebbings. Octagon is not affiliated with the show; this page is an independent summary.
