Short Answer

The highest score on Humanity's Last Exam before December 31, 2026, is expected to reach at least 55%. This outcome, priced at 98.0%, represents a marginal gain over the current official record of 54.80%.

1. Market Behavior & Drivers

The market's modest price increase to 60.0% around September 24 appears driven by reports of a new high score on Humanity's Last Exam (HLE). LLMBoard/BenchLM data shows Claude Fable 5.1 achieving a 65.0% score. This figure represents a new peak for the benchmark and likely provided the upward pressure seen in the market's recent movement from 58.0%.
However, the limited price reaction suggests traders are heavily discounting this single data point due to conflicting benchmark reports. The same model, Fable 5.1, is listed at 59.1% by Artificial Analysis. Furthermore, Scale Labs' finalized 2,500-question leaderboard shows a different leader, GPT-6 Astra, at 54.80%, while Fable 5.1 scores a much lower 46.50%. The market's current pricing near 60.0% indicates significant uncertainty and suggests a belief that the definitive high score may settle closer to the figures from Artificial Analysis or Scale Labs rather than the higher, uncorroborated LLMBoard score.
  • Since last update (~126d): Market for "At least 75%" surged +66.0pp, flipping the edge, with market_led.
  • Our model led with a +44.9pp gain for "At least 70%", widening the edge.
  • Headline market probability grew +11.0pp versus model's +9.2pp, compressing the edge.
  • For "At least 65%", our model rose +27.4pp while market fell -1.0pp, widening the edge.
  • A score of at least 55% appears very likely, exceeding the current 54.80% top.
  • Scores of 70% and 75% are significantly less likely than 60% or 65%.
  • HLE question set or scoring rubric updates are highly likely before 2027.

Who Wins and Why

Outcome Market Model Why
At least 60% 60.0% 78.0% A year-end forecast with a central estimate near 62% makes this outcome plausible.
At least 65% 38.0% 68.7% This threshold is above the year-end forecast's central estimate near 62%.
At least 55% 98.0% 96.3% The current top score of 54.80% suggests this threshold is near current performance.
At least 70% 20.0% 68.7% The current top score and year-end forecast suggest this threshold appears overly optimistic.
At least 50% 99.0% 98.1% The current highest verified score on the official Humanity's Last Exam is 54.80%.

Current Context

HLE scores show variation across reporting leaderboards. As of September 24, 2026, Claude Fable 5.1 is the reported leader on Humanity’s Last Exam (HLE), but specific scores vary significantly depending on the leaderboard version and configuration [1]. On LLMBoard/BenchLM, Fable 5.1 registers 65.0%, while Artificial Analysis lists it at 59.1% [1]. Scale Labs’ finalized 2,500-question leaderboard reports GPT-6 Astra at 54.80% and Fable 5.1 (xhigh) at 46.50% [2]. Scale Labs notes its finalized 2,500-question HLE is distinct from prior versions, making direct score comparisons difficult without identifying the benchmark release and evaluation protocol [2]. On a widely referenced group of leaderboards, Claude Fable 5.1 is listed at 59.1%, ahead of Claude Fable 5 at 55.5% and Claude Opus 5 at 54.9%; several aggregators confirmed this ordering in mid-to-late September 2026 [3]. Earlier HLE results were lower, with X-Masters (SciMaster) achieving 32.1% in July 2025, which was presented as the first result above 30% [4]. The original HLE, released in January 2025, listed o3-mini-high at 13.4% on its text-only subset [5]. HLE-Verified, updated in February 2026, increased reported model accuracy by 7–10 percentage points generally, and 30–40 points on specific items [6].
Prediction markets indicate a high probability for 50%+ by year-end. Kalshi tracks achievement thresholds ranging from 50% to 75% for resolution by December 31, 2026 [7]. Polymarket’s contract specifies a "Yes" resolution if any model reaches at least 60% on the official HLE leaderboard (agi.safe.ai) by December 31, 2026, 11:59 PM ET [8]. Market evidence strongly suggests a score of at least 50% will be reached by year-end; a Kalshi odds mirror reports 99% implied probability for ≥50%, and Octagon shows 87% market probability for ≥50% versus 7% for ≥90% [7]. Polymarket’s displayed market state for the broader 2026 outcome is roughly 70% [8]. The best forecast for the highest official score by December 31, 2026, is approximately 60–65%, with a central estimate of 62% [1]. This forecast is based on the current 59.1–65.0% reported range, anticipated frontier-model releases, and the structure of contract thresholds [1]. An earlier analyst forecast, not an observed prediction-market price, set a central estimate of 50% for the highest score by December 31, 2026, with a plausible range of 40–60%, expecting a frontier tool-augmented reasoning system to lead [5].
Model scaling drives HLE gains, but benchmark changes add uncertainty. Experts attribute recent performance gains to scaling, stronger reasoning methods, architectural innovation, and competition among key developers such as Anthropic, OpenAI, and Google DeepMind [8]. Major uncertainties include new model releases, changes to evaluation protocols, and revisions to HLE-Rolling or overall benchmark quality [8]. A July 2026 psychometric analysis found that HLE’s eight subject labels might not represent distinct latent abilities, as a general factor accounted for 98–99% of item-response variance in the multiple-choice subset [9]. This suggests caution in interpreting domain-specific rankings [9]. Expert opinion indicates HLE scores above 50% would signal expert-level closed-ended academic question answering, though not autonomous research ability or AGI; benchmark validity and item quality remain central uncertainties [5]. No retrieved source established an official prediction-market contract specifically resolving the highest HLE score by December 31, 2026 [5]. Adjacent prediction market research, such as TruthTensor’s study (December 2025–January 2026) and Prediction Arena’s findings on model performance in different market formats, suggests caution when translating AI benchmark capabilities into market probabilities [10].
Sources (10)
  1. 1Humanity's Last Exam Leaderboard | LLMBoardllmboard.ai
  2. 2Humanity's Last Exam - Scale Labs Leaderboardlabs.scale.com
  3. 3Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  4. 4SciMaster: Towards General-Purpose Scientific AI Agents Part I. X-Master as Foundation — Can We Lead on Humanity’s Last Exam?arxiv.org
  5. 5Humanity’s Last Examarxiv.org
  6. 6HLE-Verified: A Systematic Verification and Structured Revision of Humanity’s Last Examarxiv.org
  7. 7Highest score on Humanity's Last Exam before end of year? - Kalshikalshi.com
  8. 8https://polymarket.com/event/highest-score-on-humanitys-last-exam-in-2026-20260723190142922polymarket.com
  9. 9Dimensionality and Measurement Precision in HLE’s Multiple-Choice Subsetarxiv.org
  10. 10TruthTensor: Evaluating LLMs through Human Imitation on Prediction Market under Drift and Holistic Reasoningarxiv.org

2. Price Chart

Historical Price (Probability)

Outcome probability
Date

3. Significant Price Movements

Notable price changes detected in the chart, along with research into what caused each movement.

Outcome: At least 75%

📈 September 23, 2026: 73.0pp spike

Price increased from 14.0% to 87.0%

What happened: The primary driver of the purported 73.0 percentage point spike on September 23, 2026, was likely the rapid, widespread misinterpretation of a specific data point related to Humanity's Last Exam (HLE). A research paper on TUMIX+ reported "73.0% for a separate metric," explicitly clarifying it was not HLE accuracy [1]. If this 73.0% figure was conflated and disseminated, likely through social media, as an official HLE score nearing the market's 75% threshold, it would have prompted a sharp, albeit false, belief in the outcome's imminence. This interpretation aligns with findings suggesting the reported spike and its catalysts may stem from unsupported or conflated signals rather than verified leaderboard facts [2]. Social media would have served as a primary driver, accelerating the spread of this misinformation.

Outcome: At least 60%

📈 September 15, 2026: 13.0pp spike

Price increased from 45.0% to 58.0%

What happened: The available evidence does not establish a reliable 13.0-percentage-point spike on September 15, 2026, for the "At least 60%" outcome on Humanity's Last Exam [3]. Consequently, no specific social media activity, traditional news, or market structure factor can be identified as a primary driver, as the event itself is unverified by the provided sources [3]. The research indicates that while Claude Fable 5.1 achieved 65.00% by September 2026, this general progress does not align with a specific causative event on the specified date [4][5][6]. Therefore, social media was irrelevant in driving this unestablished price movement.

📈 September 12, 2026: 9.0pp spike

Price increased from 54.0% to 63.0%

What happened: The provided web research does not substantiate the reported 9.0 percentage point spike in the prediction market price on September 12, 2026, nor does it identify an associated social media or news catalyst [7][8][9][10]. While Anthropic's Claude Fable 5.1 achieved a 65% "with-tools" score on Humanity's Last Exam, this evaluation was dated September 17, 2026, occurring after the requested market movement [4][6][11]. No social media posts from key figures or traditional news announcements corresponding to a September 12 spike were found. Therefore, without evidence of the specified market event or a contemporaneous trigger, social media cannot be identified as a primary driver.

Outcome: At least 65%

📈 September 14, 2026: 11.0pp spike

Price increased from 40.0% to 51.0%

What happened: The retrieved evidence does not document a specific social media post, traditional news announcement, or market structure factor that served as the primary driver for the 11.0 percentage point spike on September 14, 2026 [10]. Although Anthropic's Claude Fable 5.1 was reported to achieve 65.0% with tools by September 18, 2026, reporting on September 14 did not identify a score above 65% [4][6][11][12][13]. Therefore, the provided information does not support the identification of a direct catalyst for this market movement. Social media's role, as well as other potential drivers, cannot be determined from the available data.

📉 September 13, 2026: 35.0pp drop

Price decreased from 75.0% to 40.0%

What happened: Research indicates that on September 13–14, 2026, traditional news reported approximately a 30-percentage-point year-over-year gain on Humanity's Last Exam (HLE) by frontier AI models, with Anthropic's Mythos Preview reaching 64.7% with tools [13]. This news, from sources like the Stanford AI Index, would typically increase the prediction market price for the "At least 65%" outcome [13]. The research explicitly notes that the September 13–14, 2026 reporting described this gain, "not a verified 35.0-point drop" [11][5][13]. Therefore, the provided information does not identify a primary driver for the stated market drop; instead, the relevant news event directly contradicts it, rendering social media activity irrelevant as a primary driver.
Sources (13)
  1. 1TUMIX: Multi-Agent Test-Time Scaling with Tool-Use Mixturearxiv.org
  2. 2HLE-Verified: A Systematic Verification and Structured Revision of Humanity’s Last Examarxiv.org
  3. 3Benchmark Radar: A Living Database and Search Engine for AI...arxiv.org
  4. 4Humanity's Last Exam Leaderboard | LLMBoardllmboard.ai
  5. 5Humanity's Last Exam Leaderboard - LLM Statsllm-stats.com
  6. 6HLE Leaderboard (September 2026): Claude Fable 5.1 Leads at 65%benchlm.ai
  7. 7Humanity's Last Exam: A Complete Guide to the AI Benchmark - AI Instant Newsaiinstantnews.com
  8. 8Next Grok Model (4.6+): Humanity’s Last Exam Debut? - Polymarket noteringen | CoinRithmcoinrithm.com
  9. 9HLE-Verified: A Systematic Verification and Structured Revision of Humanity’s Last Examarxiv.org
  10. 10HLE-Verified: A Systematic Verification and Structured Revision of Humanity’s Last Examarxiv.org
  11. 11Humanity's Last Exam Benchmark Scores & AI Model Leaderboard | BenchmarkListbenchmarklist.com
  12. 12Humanity's Last Exam Leaderboard & Results | AI Atlasai-atlas.co
  13. 13Frontier AI models gained 30 percentage points in a year on Humanity's Last Exam, a test built to defeat themspacemart.com

4. Market Data

Contract Snapshot

This market will resolve to YES if the highest score on "Humanity's Last Exam" is 65% or greater. It will resolve to NO if the highest score is less than 65%. The resolution will be determined by December 31, 2026.

Available Contracts

Market options and current pricing

Outcome bucket Yes (price) No (price) Last trade probability
At least 50% $1.00 $0.03 99%
At least 55% $0.99 $0.06 98%
At least 75% $0.75 $0.95 87%
At least 60% $0.74 $0.54 60%
At least 80% $0.12 $0.97 53%
At least 65% $0.56 $0.59 38%
At least 70% $0.59 $0.81 20%
At least 90% $0.20 $0.99 9%
At least 85% $1.00 $1.00 4%

Market Discussion

As of September 24, 2026, the highest publicly listed Humanity's Last Exam score is 59.1% by Anthropic's Claude Fable 5.1, with other Claude variants and GPT-6 Astra following closely [1][2][3][4][5]. Prediction market sentiment positions Anthropic as the current favorite, with expectations for continued rapid score growth, potentially reaching around 65% by December 31, 2026, through new model releases and enhanced reasoning [6][7][8][9][10][11][12][13][14][15][16]. However, comparisons are complicated by differing leaderboard versions and weakening measurement precision at the frontier [2][17][18][16].

Sources (18)
  1. 1Humanity's Last Exam Benchmark Leaderboard | Artificial Analysisartificialanalysis.ai
  2. 2Humanity's Last Exam - Scale Labs Leaderboardlabs.scale.com
  3. 3Humanity's Last Exam Leaderboard & Results | AI Atlasai-atlas.co
  4. 4Humanity's Last Exam Leaderboard 2026 - Compare AI Model Scorespricepertoken.com
  5. 5Humanity's Last Exam (AA) leaderboard: 544 AI models ranked | BenchLeaderbenchleader.com
  6. 6https://polymarket.com/event/highest-score-on-humanitys-last-exam-in-2026-20260723190142922polymarket.com
  7. 7https://polymarket.com/event/highest-google-gemini-score-on-humanitys-last-exam-in-2026-20260723192605545polymarket.com
  8. 8https://polymarket.com/event/highest-meta-score-on-humanitys-last-exam-in-2026-20260723192006805polymarket.com
  9. 9https://polymarket.com/pt/event/highest-claude-score-on-humanitys-last-exam-in-2026-20260723190836285polymarket.com
  10. 10https://polymarket.copilot.markets/event/highest-claude-score-on-humanitys-last-exam-in-2026-20260723190836285polymarket.copilot.markets
  11. 11https://polymarket.copilot.markets/id/event/highest-openai-score-on-humanitys-last-exam-in-2026-20260723225144062polymarket.copilot.markets
  12. 12Will the highest score achieved by a Moonshot Kimi model on Humanity’s Last Exam in 2026 be 45% or higher? | Paritypredictparity.com
  13. 13[2607.20268v1] PoTRE: Test-Time Reasoning inspired by Cognitive Heterogeneityarxiv.org
  14. 14SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?arxiv.org
  15. 15[2508.02789v1] Cognitive Loop via In-Situ Optimization: Self-Adaptive Reasoning for Sciencearxiv.org
  16. 16[2609.27176] Beyond Overlap: Estimating the Causal Effect of Benchmark Exposurearxiv.org
  17. 17GPT-6 Astra: A new generation of intelligence - OpenAIopenai.com
  18. 18Dimensionality and Measurement Precision in HLE’s Multiple-Choice Subsetarxiv.org

5. Trust Index

This scorecard was computed Nov 7, 2026 and may not reflect current market conditions.

Octagon Trust Index Kalshi 74 Good

Primary risk· Trade quality

How it adds up
Integrity80% of score81Good
Trade quality20% of score47High Risk
Trust score74Good

Weighted blend with hard caps — a critically weak safety pillar, or a severe trading anomaly, caps the total regardless of the rest. Full methodology · About the Trust Index

Trust profile
Integrity4 screens run · 6 don't apply

6. What is the likelihood of Scale AI updating the Humanity's Last Exam question set or scoring rubric before 2027, and how could such changes impact model scores?

Likelihood of HLE update by 202780–90% [1][2][3]
Probability of fixed leaderboard replacement20–35% [1][2][3]
Potential score impact from changesSeveral percentage points (up or down) [2][3][4]
Scale AI is highly likely to update Humanity's Last Exam before 2027. There is an 80–90% chance of some benchmark-version change occurring, a prediction based on repeated past updates and the explicit maintenance of dynamic variants like HLE-Rolling and HLE-Diamond [1][2][3]. The HLE materials themselves describe ongoing processes for removing or replacing bug-bounty-flagged and searchable questions, driven by reported concerns about answer correctness [2][3][5]. However, the probability of a fixed official leaderboard, particularly one used by a market, being retroactively replaced is considerably lower, estimated at 20–35% [1][2][3].
HLE revisions can significantly alter benchmark difficulty and model scores. Such changes are not merely theoretical; they can introduce score discontinuities, potentially shifting model scores up or down by several percentage points rather than providing a predictable boost [2][3][4]. Replacing questions, or changing the scoring rubric or the LLM equality checker, can mechanically alter scores. For instance, harder replacement sets might lower scores, while easier or more model-familiar questions could raise them. Even without a change in model behavior, score shifts can occur, particularly for short-answer, mathematical, ambiguous, or numerically tolerant responses [1][2][6].
Prediction market resolution depends on the specific HLE version used. For these markets, resolution is typically tied to the official HLE accuracy or leaderboard metric [4]. Therefore, a version change only becomes relevant if the market's resolution rules consider the revised version as the pertinent HLE result. Separately labeled benchmarks like HLE-Rolling or HLE-Diamond may not automatically count towards the official market resolution [4]. The primary risk for prediction markets lies in potential ambiguity over which specific HLE version the official leaderboard and market resolution will ultimately utilize [2][3][4].
Sources (6)
  1. 1Humanity's Last Exam - Scale Labs Leaderboardlabs.scale.com
  2. 2Introducing HLE-Diamond | Humanity's Last Examlastexam.ai
  3. 3Humanity's Last Examlastexam.ai
  4. 4Highest score on Humanity's Last Exam before Dec 31, 2026? - Kalshi Odds | CoinRithmcoinrithm.com
  5. 5Humanity's Last Exam - Wikipediaen.wikipedia.org
  6. 6Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai

7. How does Scale AI's official 2,500-question HLE leaderboard methodology differ from other aggregators, and what is the current verified top score?

Official HLE Top Score54.80% 1.94 pp (GPT 6 Astra, Sep 24, 2026) [1]
Official HLE Question Count2,500 questions [1]
Prediction Market Threshold60% accuracy by Dec 31, 2026 [1][2]
Scale AI's HLE leaderboard employs a rigorous, distinct evaluation methodology. The official methodology utilizes exactly 2,500 questions, separate from the earlier 3,000-question "HLE-preview" release [1]. This dataset construction involves expert review by Scale Outlier reviewers with graduate degrees, standardized rubrics, and multi-stage approval [1]. Scale's ranking method is not a raw-score sort; instead, it ranks by "Rank (Upper Bound)," identifying models statistically significantly better than a target by requiring the better model's 95% confidence interval lower bound to exceed the target's 95% confidence interval upper bound [1]. The official evaluation runs every model on all public HLE questions at temperature 0.0 when configurable, prompting for a final answer plus confidence, and reports accuracy with uncertainty and calibration error [1]. As of September 24, 2026, the verified top score on this official leaderboard for the finalized HLE is 54.80% 1.94 percentage points, achieved by GPT 6 Astra [1]. This score is 5.20 percentage points below the 60% accuracy threshold required for a prediction market to resolve "Yes" before December 31, 2026, based on the official HLE source [1][2].
Other aggregators show differing scores due to varied methodologies and subsets. In contrast, aggregators, such as Artificial Analysis, evaluate a reduced subset, like 2,158 text-only HLE questions [3][4]. This leads to different displayed results, for instance, Artificial Analysis reports 59.1% [3][4]. Aggregator numbers can vary due to factors including question coverage, prompts, tools, grading harnesses, and versioning [5].
Sources (5)
  1. 1Humanity's Last Exam - Scale Labs Leaderboardlabs.scale.com
  2. 2Highest score on Humanity's Last Exam before Dec 31, 2026? - Kalshi Odds | CoinRithmcoinrithm.com
  3. 3Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  4. 4Artificial Analysis Intelligence Benchmarking Methodologyartificialanalysis.ai
  5. 5Humanity's Last Exam Leaderboard - LLM Statsllm-stats.com

8. How do the architectural approaches of Anthropic's Claude 5 series and OpenAI's GPT-6 series compare in their ability to solve the complex reasoning problems on Humanity's Last Exam?

Claude Fable 5.1 HLE Score59.1% and 65.0% (September 24, 2026 [1][2])
Claude Opus 5 HLE Score64.7% (September 24, 2026 [1][2])
GPT-6 Astra HLE Score (reported)54.7% (September 24, 2026 [1])
Anthropic's Claude 5 and OpenAI's GPT-6 series employ distinct architectural approaches. The Claude 5 series, encompassing Opus 5, Fable 5, and Mythos 5, emphasizes adaptive reasoning, long-context processing, and tool-agent workflows [3][4][5]. Opus 5, in particular, has demonstrated improvements in long-horizon agentic behavior and mathematical/scientific reasoning [3][4][5]. Conversely, OpenAI's GPT-6 Astra is designed with a focus on broad reinforcement learning, multimodal browsing, and computer-use competence [3][6]. Humanity's Last Exam (HLE) serves as a critical benchmark for evaluating these models, although achieving direct comparisons remains difficult due to variations in tool access, reasoning budget, fallback behavior, and benchmark version [3][7][1][2][8][9][10].
Claude 5 series models consistently demonstrate strong performance on Humanity's Last Exam. Public leaderboards from September 24, 2026, indicate Claude Fable 5.1 achieved scores of 59.1% and 65.0%, while Claude Opus 5 reached 64.7% [1][2]. The Claude 5 family, with Fable 5.1 specifically noted, is presently considered a safer leader and probable high-score holder in prediction markets [7][11][1][2][8]. The Opus 5 system card specifically references HLE performance as proof of enhanced agentic search and reasoning capabilities [12].
GPT-6 Astra shows promising frontier capabilities, despite varied HLE benchmark results. As a component of the diverse GPT-6 lineup, it is positioned as a frontier capability model, exhibiting robust performance in benchmarks such as FrontierMath Tier 4 (98%) and ARC-AGI-3 (99.9%) [3][6]. Regarding Humanity's Last Exam, one report from September 24, 2026, indicated GPT-6 Astra achieved 54.7% [3][1]. However, its official announcement presented a wider spectrum of HLE scores, ranging from 57.2% to 65.0%, without adequate contextual details regarding specific test configurations [3][1]. Notwithstanding these disparate HLE snapshots, GPT-6 Astra is regarded as a credible potential upset candidate, attributed to its recent introduction and assertions of frontier reasoning abilities, though a precise probability cannot be responsibly determined [7][11][1][2][8].
Sources (12)
  1. 1Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  2. 2Humanity's Last Exam Leaderboard 2026 - Compare AI Model Scorespricepertoken.com
  3. 3GPT-6 Astra: A new generation of intelligence | OpenAIopenai.com
  4. 4Introducing Claude Opus 5 - Anthropicanthropic.com
  5. 5Claude Fable 5 and Claude Mythos 5 - Anthropicanthropic.com
  6. 6Introducing GPT‑6 Sol and Luna - OpenAIopenai.com
  7. 7Humanity's Last Examagi.safe.ai
  8. 8Humanity's Last Exam Leaderboard - LLM Statsllm-stats.com
  9. 9Introducing Claude Opus 4.6 - Anthropicanthropic.com
  10. 10Humanity's Last Exam - arXiv.orgarxiv.org
  11. 11OpenAI’s Astra Model: Humanity’s Last Exam Debut?polymarket.com
  12. 12Claude Opus 5 System Card | alphaXivalphaxiv.org

9. Which public sources provide the most reliable data for tracking performance on the official Humanity's Last Exam leaderboard ahead of the December 31, 2026 deadline?

Official Leaderboardlastexam.ai/leaderboard and labs.scale.com/leaderboard/humanitys_last_exam [1][2]
Official HLE Questions2,500 questions across more than 100 subjects [3][4][5]
Ranking MetricRank (Upper Bound) based on 95% confidence intervals [1]
The official leaderboard provides the most reliable HLE performance data. The most accurate data for tracking performance on the official Humanity's Last Exam (HLE) leaderboard is found directly on the official HLE/Scale Labs leaderboard, accessible at lastexam.ai/leaderboard and labs.scale.com/leaderboard/humanitys_last_exam [1][2]. This Scale Labs page explicitly identifies itself as the 'Humanity's Last Exam - Scale Labs Leaderboard' and details the current finalized 2,500-question HLE version [1]. For precise tracking, it is recommended to archive the official Scale Labs/lastexam.ai leaderboard at each observation point, recording the model name, score, version, date, and confidence interval [1].
HLE benchmark features 2,500 questions and uses a private test set. The HLE benchmark consists of 2,500 questions across more than 100 subjects, including both multimodal and text questions, and employs a private held-out test set to prevent public-set overfitting [3][4][5]. The Scale leaderboard ranks models using Rank (Upper Bound), which is based on 95% confidence intervals; a model is deemed statistically superior only if its lower confidence bound exceeds another model's upper bound [1]. To verify any changes to the benchmark version, the official HLE site and paper, located at agi.safe.ai and lastexam.ai/paper, should be consulted [3][5].
Third-party trackers and variant HLE scores require careful distinction. While third-party trackers such as Artificial Analysis, PricePerToken, and Sophon can be utilized for discovery and timestamped cross-checks, their data should not override the official leaderboard [6][7][8]. It is crucial to differentiate HLE-Diamond, a distinct 1,000-question subset released on September 22, 2026, and its scores should not be combined with the official finalized 2,500-question HLE score unless market resolution rules specifically allow it [9]. Similarly, the Hugging Face HLE agent leaderboard represents a separate evaluation track for agents equipped with tools and is not automatically comparable to the official closed-book/model leaderboard [10]. Independent or tool-enabled leaderboards should be considered non-resolving unless explicitly incorporated by the market rules [3][1][6][7][8].
Sources (10)
  1. 1Humanity's Last Exam - Scale Labs Leaderboardlabs.scale.com
  2. 2Humanity's Last Examlastexam.ai
  3. 3Humanity's Last Examagi.safe.ai
  4. 4Humanity's Last Examlastexam.ai
  5. 5[2501.14249] Humanity's Last Exam - arXivlastexam.ai
  6. 6Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  7. 7Humanity's Last Exam Leaderboard 2026 - Compare AI Model Scorespricepertoken.com
  8. 8Humanity's Last Exam (HLE) benchmark: model scores & leaderboardsophon.at
  9. 9Introducing HLE-Diamond | Humanity's Last Examlastexam.ai
  10. 10HLE Leaderboard for Agents with Tools - a Hugging Face Space by...huggingface.co

10. What has been the historical rate of improvement for frontier models on Humanity's Last Exam and comparable reasoning benchmarks over the last 18 months?

Initial HLE Score9.4% (DeepSeek-R1 in early 2025) [1][2]
HLE Score by Sept 202665% (Claude Fable 5.1) [3][4]
HLE Score Increase (1 year)Approximately 30 percentage points (over past 12 months) [5]
Frontier models have significantly advanced on Humanity's Last Exam (HLE). Over the last 18 months, frontier models have demonstrated substantial progress on Humanity's Last Exam (HLE) and similar reasoning benchmarks. Top scores on HLE increased from a low baseline of 9.4% achieved by DeepSeek-R1 in early 2025 to 65% by September 2026 [5][3][4][1][2]. This represents an improvement of over 55 percentage points during this period.
Performance on HLE has shown consistent upward trends. Over approximately the past year, Stanford's 2026 AI Index reported that frontier models gained about 30 percentage points on HLE [5]. Publicly reported milestones indicate scores around 44.7–46.9% by May 2026, which translates to an increase of about 35–40 points from the early 2025 reference [5]. By mid to late 2026, third-party leaderboards showed HLE scores in the range of 59.1–65%, with Claude Fable 5.1 notably reaching 65% in September [3][6][7][4].
HLE offers more improvement scope than other benchmarks. While comparable reasoning benchmarks such as GPQA Diamond and AIME 2025 show top models nearing saturation at 94–96%, HLE has retained more room for further improvement [6][8][7]. It is also important to note that various factors, including the dataset version, modality, tools utilized, and prompting methods, can materially influence the apparent rate of score growth on these benchmarks [3][6][7].
Sources (8)
  1. 1Humanity's Last Examagi.safe.ai
  2. 2A benchmark of expert-level academic questions to assess... - Naturenature.com
  3. 3Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  4. 4HLE Leaderboard (September 2026): Claude Fable 5.1 Leads at 65%benchlm.ai
  5. 5Technical Performance | The 2026 AI Index Report | Stanford HAIhai.stanford.edu
  6. 6Large Language Model Benchmarks: A Taxonomy of Capabilities...mdpi.com
  7. 7LLM Benchmarks Compared: MMLU, HumanEval, GSM8K and More...lxt.ai
  8. 8AI Benchmark Guide | DemandSpheredemandsphere.com

11. What Could Change the Odds

Key Catalysts

Scheduled or official frontier-model releases, new FrontierMath and Open Problems additions with verified machine-assisted proofs, and benchmark-result updates are key bullish AI/science prediction-market catalysts before December 31, 2026 [1][2]. Polymarket describes official announcements, deadlines, data, and major news as primary market catalysts [2]. Claude Fable 5.1 leads the Humanity's Last Exam (HLE) leaderboard at 59.1% on the Artificial Analysis evaluation, a result independently reproduced by ModelCap and BenchmarkList [3][4][5]. Frontier models gained roughly 30 percentage points on HLE in one year, illustrating rapid capability progress and a risk that the benchmark saturates quickly [6]. The highest displayed HLE score reported by Benchmark Radar as of September 13, 2026, was 55.47% [7]. HLE-Verified materially changes measured scores, with verification gains as large as 39.58 percentage points for DeepSeek-V3.2 [8].
Market contract resolution dates indicate potential late-2026 catalysts, including September 30 for public availability of OpenAI Astra, October for GPT-6/Astra-related releases, and December 31 for GPT-6 [9][10]. These are market-resolution deadlines or contract framing, not independently confirmed company launch dates [9][10][11]. A major model launch, a verified frontier-math solve, or a new benchmark-leading result before year-end could raise expectations for further HLE gains and AI-science markets [1][10][11]. Conversely, safety/red-team delays, regulatory friction, inaccessible or private releases, benchmark-version ambiguity, or failure to verify claimed proofs could reverse pricing [1][10][11].
Contract resolution is a key risk, as rule version, exact official release object, correction history, and unfinished conditions can alter when an event is resolvable [12]. The timeline for observation suggests monitoring official release announcements and leaderboard refreshes through September 30, then model launches and FrontierMath/Open Problems verification in October, and finally benchmark submissions, independent replication, safety/regulatory decisions, and contract-resolution evidence from November through December 31 [3][4][6][1][9][10]. All implied market probabilities should be treated as dynamic rather than forecasts [1][10][11]. The HLE benchmark is intentionally difficult, and small, near-zero changes are not strong evidence of capability progress; thus leaderboard comparisons require version, subset, modality, and verification-status controls [13][14].

Key Dates & Catalysts

  • Strike Date: December 31, 2026
  • Expiration: December 31, 2026
  • Closes: December 31, 2026
Sources (14)
  1. 1Another Frontier Math Problem Solved by October 31? Odds | Lines.comlines.com
  2. 2Technology Prediction Markets & Live Odds 2026 | Polymarketpolymarket.com
  3. 3Humanity's Last Exam Benchmark Leaderboard - Artificial Analysisartificialanalysis.ai
  4. 4AA Humanity's Last Exam leaderboard: Claude Fable 5.1 leads 185 modelsmodelcap.ai
  5. 5Humanity's Last Exam Benchmark Scores & AI Model Leaderboard | BenchmarkListbenchmarklist.com
  6. 6Technical Performance | The 2026 AI Index Report | Stanford HAIhai.stanford.edu
  7. 7Benchmark Radar: A Living Database and Search Engine for AI...arxiv.org
  8. 8HLE-Verified: A Systematic Verification and Structured Revision of Humanity’s Last Examarxiv.org
  9. 9OpenAI’s Astra released by…?polymarket.com
  10. 10GPT-6 Released by…? OpenAI’s 2026 Deadline Market | Lines.comlines.com
  11. 11Next Claude Opus released by...?polymarket.com
  12. 12[2609.16642] From Public Evidence to Contractual Outcome: First and Stable Decidability on Kalshiarxiv.org
  13. 13Humanity’s Last Examarxiv.org
  14. 14[2501.14249] Humanity's Last Exam - arXivarxiv.org

13. Related News