Episode summary
Understanding the Most Viral Chart in Artificial Intelligence | Odd Lots
Odd Lots
· · Hosts: Tracy Alloway, Joe Weisenthal · Guests: Chris Painter, Joel Becker
Summary
The episode examines METR’s “time horizon” charts, which try to measure AI progress by asking how long it would take a human to do the same tasks that a model can complete with a 50% success rate. The guests argue that the viral chart is not showing an AI running continuously for 12 hours, but rather that the model can solve tasks whose human completion time is about 12 hours, on average, under controlled conditions. They say the clearest signal in the chart is an exponential rise in capability, with recent models roughly doubling their effective time horizon every few months.
The discussion also focuses on what METR is actually trying to do: provide evidence about AI autonomy, misalignment risk, and the point at which systems may become capable of doing long, complex actions on their own. The guests say their work is concentrated in software engineering and machine-learning tasks because those are the most relevant to early AI R&D automation and are easier to benchmark reliably than more open-ended human work.
They also address criticism that the chart can be overread. The guests say the 50% threshold is statistically convenient and historically grounded, but they agree that real-world productivity gains are messier because tasks are harder, more collaborative, and harder to score automatically. Even so, they argue the underlying trend is real and important enough that investors, labs, and policymakers should treat it as a serious signal.
Key takeaways
- The guests say METR’s chart measures task difficulty in human-hours, not wall-clock time spent by the model.
- They argue the recent frontier models have been on an exponential trajectory, with doubling times of about four to seven months.
- The guests say software engineering and machine-learning tasks are prioritized because they are most relevant to AI R&D automation and early warning signs.
- They acknowledge that a 50% benchmark can overstate real-world usefulness, because production work is messier and requires verification.
- The guests say there is tension between safety and commercial pressure, especially when companies make large compute and data-center commitments.
Notable quotes
“we are plotting the difficulty of tasks the AIs are able to complete over time”
“we predict that it will succeed at those tasks around 50% of the time”
“I think the the reason that we work on things like the time horizon charts is because if we're trying to establish the stakes”
How the episode opens
Bloomberg Audio Studios. Podcasts, radio, news. — Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal. And I'm Tracy Alloway. Tracy, one thing about AI is that uh lots of lines that go up. — Uh yes, famously there is perhaps one line that has captured the attention more than others when it comes to lines going up. Yes, but we're recording this April 7th. Did you see the uh Anthropic revenue chart, by the way? It's just extreme. It's just on the number of lines going up. I mean, there are some some really I just Let me caveat that.
Up until recently, there was one chart of a line going up exponentially that became, I think it's fair to say, the most viral chart in AI, right? Yes, I would absolutely agree with that. So, one of the many lines that go up, and there are various lines that sort of capture this, is um essentially just measures of AI progress and what they could do.
What the models are capable of and so forth, and you know, there's all different benchmarks out there and hobbyist benchmark creators, etc. All kinds of benchmarks out there. Uh organization called Meter, based out in San Francisco, and they measure how well AI models are doing sort of engineering tasks, etc.
And they have these charts showing how long, you know, certain tasks, how long it would take a human to do them, and then whether AI could do them, and yes, the lines just almost vertical. I think there was someone like one of the ones that came out maybe very early this year or late last year showing the latest Claude model, and it was just like, this is crazy.
From the episode’s captions: roughly the first two minutes, lightly cleaned. Keep watching on YouTube →
Topics
- ai model evaluation
- autonomous task benchmarks
- viral ai capability charts
- recursive self improvement risk
- threat research
Companies and tickers mentioned
- METR
Watch the episode
Watch “Understanding the Most Viral Chart in Artificial Intelligence | Odd Lots” on YouTube
Summary, takeaways and quote selection by Octagon. Quotes are short excerpts; the full conversation is in the episode.
More from Odd Lots
-
How a Chicago Restaurant Has Stayed Open For Nearly Three Decades | Odd Lots -
How Airlines Actually Hedge Higher Fuel Prices | Odd Lots -
Robert Friedland on the World's Monumental Shortage of Copper | Odd Lots -
Brad Setser on the US's Unusual Japanese Yen Intervention | Odd Lots -
The Tungsten Market Is Warning of an Upcoming War | Odd Lots -
These Are the Sharps Actually Making Money on Prediction Markets | Odd Lots
Odd Lots, this episode and its artwork belong to Bloomberg. Octagon is not affiliated with the show; this page is an independent summary.
