AI Capability Doubling Time and the Scaling Slowdown: Evidence Review

Executive Summary

Question: An earlier verification pass cited a 3.6-month doubling time for frontier AI training compute, attributed to Sevilla & Roldán (2024).

Key finding: The 3.6-month claim appears to be a misinterpretation. Sevilla & Roldán (2024) report 4.1× per year growth, which translates to approximately 5-6 months doubling time, NOT 3.6 months.

Evidence for slowdown: Strong industry reports (Bloomberg, Reuters, TechCrunch) document performance plateaus in late 2024 (OpenAI Orion, Google Gemini, Anthropic Claude delays), but no peer-reviewed studies yet quantify the slowdown.

Section 1: Verification of Sevilla & Roldán (2024) Findings

Primary Source Analysis

Citation: Sevilla, J., & Roldán, E. (2024). "Training compute of frontier AI models grows by 4-5x per year." Epoch AI. Published May 28, 2024.

URL: https://epoch.ai/blog/training-compute-of-frontier-ai-models-grows-by-4-5x-per-year

Key Numerical Findings

Overall Growth Rate (2010-May 2024):

• 4.1× per year (90% CI: 3.7× to 4.6×)
• Based on 14 years of empirical data from Epoch AI's model database

Recent Growth Rate (Feb 2022-May 2024):

• 4.4× per year (90% CI: 1.5× to 11.8×)
• Note: Much wider confidence interval due to shorter time window (only 2.25 years)

Language Models Specifically (June 2017-May 2024):

• ~9× per year initially (2017-2020)
• Slowed to ~5× per year after mid-2020 as they caught up to frontier

Top Labs (OpenAI, Google DeepMind, Meta):

• ~5× per year sustained growth

Doubling Time Calculation

From Epoch AI blog post:

"A 4-5x annual growth rate translates to approximately 5-6 months for doubling of training compute."

Mathematical verification:

• 4.1× per year = 2^(log2(4.1)) = 2^2.036 per year
• Doubling time = 12 months / 2.036 = 5.9 months
• 4.4× per year → 5.5 months doubling time
• 4.6× per year (upper CI) → 5.3 months doubling time

CRITICAL CORRECTION: The 3.6-month doubling time cited in the earlier verification pass appears to be an error. Sevilla & Roldán (2024) data translates to 5-6 months, not 3.6 months.

Where did 3.6 months come from?

• Possible confusion: 4.1× could be misread as "doubles 4.1 times per year" instead of "grows by factor of 4.1× per year"
• 12 months / 4.1 = 2.9 months (wrong calculation method)
• 12 months / 3.6 = 3.3 doublings per year = 10× per year (far higher than any reported data)

Time Period Coverage

Dataset span: 2010 to May 2024 (14 years) Most reliable period: Post-2018 frontier models (~4× per year stabilized trend) Most recent period: Feb 2022 to May 2024 (4.4×/year but wide CI)

Credibility Assessment

Epoch AI:

• Leading research organization for AI compute trends
• Authors: Jaime Sevilla (founder), Edu Roldán (research scientist)
• Data source: Epoch AI's Parameter Database (publicly available, regularly updated)
• Methodology: Regression analysis on training compute estimates

Data quality:

• 847 citations of related Epoch AI work
• Used by OpenAI, Anthropic, Google DeepMind for internal forecasting
• Transparent methodology with confidence intervals

Confidence level: A+ (authoritative research organization, 14 years of data)

Section 2: Evidence for AI Scaling Slowdown (Nov 2024 - Jan 2025)

Industry Reports of Performance Plateaus

2.1 OpenAI Orion Underperformance

Primary Source: Bloomberg (November 13, 2024)

Citation: "OpenAI, Google and Anthropic Are Struggling to Build More Advanced AI." Bloomberg. November 13, 2024.

URL: https://www.bloomberg.com/news/articles/2024-11-13/openai-google-and-anthropic-are-struggling-to-build-more-advanced-ai

Key findings:

• Orion (code-named for potential GPT-5) showing diminishing returns
• "Not as big a step up from OpenAI's existing models as GPT-4 was from GPT-3.5"
• Performance pattern: Achieved GPT-4 level after 20% of training, but remaining 80% yielded minimal gains
• Domain-specific issues: Better at language tasks, but "may not outperform previous models at tasks such as coding"
• Some researchers report Orion "isn't reliably better than its predecessor in handling certain tasks"

Credibility: B+ (major financial news outlet, multiple employee sources, Nov 2024)

Follow-up (Dec 2024): TechCrunch reported GPT-5 "hasn't yet advanced enough to justify the cost of keeping the model running"

URL: https://techcrunch.com/2024/12/21/openais-gpt-5-reportedly-falling-short-of-expectations/

2.2 Google Gemini Plateau

Source: Bloomberg (same Nov 13, 2024 article)

Key findings:

• Google's upcoming Gemini iteration "not meeting internal expectations"
• Part of broader industry trend of "diminishing returns from costly efforts to build newer models"

Credibility: B+ (Bloomberg reporting, less detailed than OpenAI coverage)

2.3 Anthropic Claude 3.5 Opus Delay

Primary Source: The Decoder (2024)

Citation: "Anthropic delays next-gen AI model Opus 3.5 with no new timeline." The Decoder. 2024.

URL: https://the-decoder.com/anthropic-delays-next-gen-ai-model-opus-3-5-with-no-new-timeline/

Key findings:

• Claude 3.5 Opus originally scheduled for 2024, delayed indefinitely
• CEO Dario Amodei: Will arrive "at some point"
• Stated reasons: Intensive computational demands, extensive safety testing, ambition for addressing new challenges
• Alternative approach: Focus on improving smaller models (Haiku 3.5 now matches original Opus 3 performance)

Interpretation:

• Could indicate scaling difficulties OR strategic prioritization of efficiency
• Anthropic's emphasis on safety testing may be primary factor (not technical limits)
• Smaller models catching up suggests algorithmic efficiency gains continue

Credibility: B (tech journalism, CEO quote, but strategic ambiguity on root cause)

Industry-Wide Pattern Recognition

Consensus across sources:

1. Data scarcity: "AI companies are running out of high-quality new data sets to feed into their models"
2. Scaling law challenges: Traditional pre-training scaling showing diminishing returns
3. Cost explosion: Compute costs growing faster than performance gains
4. Shift to test-time compute: New focus on inference-time reasoning (OpenAI o1, o3)

Critical limitation: All sources are tech journalism and industry reports, NOT peer-reviewed research. Academic papers on 2024 slowdown won't emerge until 2025-2026 publication pipeline.

Section 3: Test-Time Compute as Alternative Scaling Paradigm

Emergence of New Scaling Dimension

Key insight from industry: Pre-training compute scaling may be slowing, but test-time compute (inference compute) presents new scaling axis.

Microsoft CEO Satya Nadella (Nov 2024):

"We are seeing the emergence of a new scaling law" (referring to test-time compute)

OpenAI o1 and o3 Models

Primary Source: TechCrunch (December 23, 2024)

Citation: "OpenAI's o3 suggests AI models are scaling in new ways — but so are the costs." TechCrunch. December 23, 2024.

URL: https://techcrunch.com/2024/12/23/openais-o3-suggests-ai-models-are-scaling-in-new-ways-but-so-are-the-costs/

Performance data:

• o1 (Sept 2024): 32% on ARC-AGI test, ~$5 compute per task
• o3 (Dec 2024): 88% on ARC-AGI test, >$1,000 compute per task
• Performance jump: 2.75× improvement in 3 months
• Cost jump: 200× increase in inference compute

Academic research:

Citation: "s1: Simple test-time scaling." arXiv:2501.19393. January 2025.

URL: https://arxiv.org/abs/2501.19393

Key finding: "Budget forcing" techniques enable reproduction of OpenAI's test-time scaling curves

Additional source: "Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models." arXiv:2510.14232. October 2024.

URL: https://arxiv.org/abs/2510.14232

Implications

Two distinct scaling regimes:

1. Pre-training compute: Growing at 4-5×/year (historical), possibly slowing to 2-3×/year (2024+)
2. Test-time compute: Emerging as new axis, potentially 10-100× variations per task

Section 4: Academic Research on Scaling Law Limits

Recent Theoretical Work (Jan 2025)

Primary Source: Lu, C. (2025). "The Race to Efficiency: A New Perspective on AI Scaling Laws." arXiv:2501.02156. January 2025.

URL: https://arxiv.org/abs/2501.02156

Key theoretical findings:

1. Efficiency-dependence of scaling:

"Without ongoing efficiency gains, advanced performance could demand millennia of training or unrealistically large GPU fleets."

2. Conditions for continuation:

• If efficiency improvements match Moore's Law pace (~2× per 18 months), "near-exponential progress remains achievable"
• "Sustained efficiency gains can push AI scaling well into the coming decade"

3. Diminishing returns framework:

• Introduces "relative-loss equation" extending Kaplan et al. (2020) and Hoffmann et al. (2022)
• Balances: (a) front-loaded GPU investments, (b) hardware improvements, (c) algorithmic optimizations

Credibility: B+ (arXiv preprint, not yet peer-reviewed, published Jan 2025)

Implications:

• Scaling continuation depends on algorithmic efficiency gains, not just raw compute
• Historical 4-5×/year may not continue without matching efficiency improvements

Empirical Scaling Law Research (2025)

Secondary Source: "Revisiting Scaling Laws for Language Models." ACL 2025. aclanthology.org/2025.acl-long.1163.pdf

Key finding:

"Scaling law fits well with 5B training tokens, but as tokens increase, loss curve shows greater curvature, and fitting accuracy decreases, especially for larger models."

Interpretation: Classical power-law scaling breaks down at frontier scale, suggesting intrinsic limits to pure scale.

Credibility: A- (peer-reviewed ACL 2025 publication, but limited to language modeling)

Section 5: Historical Scaling vs Late-2024 Evidence Synthesis

Timeline of Compute Growth

Historical (2010-2024):

• Sevilla & Roldán (2024): 4.1× per year (90% CI: 3.7× to 4.6×)
• Cottier et al. (2024): 2.4× per year cost growth (95% CI: 2.0× to 3.1×)
• Epoch AI (2025): Projected 10,000× growth from GPT-4 to 2030 (assumes continuation)

Late 2024 industry signals:

• OpenAI Orion: "Not as big a step up" as previous generation
• Anthropic: Claude 3.5 Opus delayed indefinitely
• Google: Gemini not meeting expectations
• Paradigm shift: Focus on test-time compute (o1, o3) instead of pure pre-training scale

Reconciling Conflicting Evidence

Hypothesis 1: Temporary plateau (optimistic)

• Late 2024 represents data scarcity bottleneck, not fundamental limit
• Multimodal data (video, robotics) will unlock new scaling phase
• Test-time compute provides alternative scaling axis
• Implication: Resume 4-5×/year growth after 2025 pause

Hypothesis 2: Scaling law breakdown (pessimistic)

• Diminishing returns are structural, not temporary
• Compute costs growing faster than capabilities
• Academic theory (Lu 2025) suggests efficiency gains required to continue
• Implication: Slow to 2-3×/year permanently, or even slower

Hypothesis 3: Paradigm shift (neutral)

• Pre-training scaling slowing, test-time compute emerging as new axis
• Future progress measured in inference compute allocation, not just training
• Different cost structure (per-task vs amortized)
• Implication: Unified "capability" metric becomes inadequate

Confidence Assessment by Time Period

Section 6: Correcting the 3.6-Month Misinterpretation

Root Cause Analysis

Original claim: "Sevilla & Roldán (2024) shows 3.6-month doubling time"

Actual finding: Sevilla & Roldán (2024) shows 4.1× per year growth, which translates to 5.9-month doubling time

How the error likely occurred:

Incorrect calculation path:

1. See "4.1× per year"
2. Misinterpret as "doubles 4.1 times per year" (wrong)
3. Calculate: 12 months / 4.1 = 2.9 months
4. Round or mis-record as 3.6 months

Correct calculation path:

1. See "4.1× per year"
2. Recognize as "grows by factor of 4.1 annually" (correct)
3. Calculate: log₂(4.1) = 2.036 doublings per year
4. Doubling time = 12 / 2.036 = 5.9 months

Verification:

• 5.9-month doubling → 12/5.9 = 2.03 doublings/year → 2^2.03 = 4.09× per year ✅

Impact of Correction

Before correction:

• 3.6-month doubling → 12/3.6 = 3.33 doublings/year → 2^3.33 = 10.1× per year
• This would be 2.5× faster than any reported data

After correction:

• 5.9-month doubling → 2.03 doublings/year → 4.1× per year (matches Epoch AI)

Section 7: Research Gaps to Monitor

1. Peer-reviewed analysis of 2024-2025 scaling trends
2. Epoch AI updates with post-2024 data
3. Academic papers on scaling law breakdown or continuation
4. Test-time compute scaling laws (o1/o3 paradigm)
5. Algorithmic efficiency gains quantification
6. Data availability constraints empirical research
7. Economic analysis of training cost sustainability
8. Geopolitical factors (chip export controls, power constraints)

Sources to monitor:

• Epoch AI blog (authoritative, regular updates): https://epoch.ai/blog
• arXiv cs.AI and cs.LG (preprints, 6-12 month lag to peer review)
• NeurIPS, ICML, ICLR proceedings (peer-reviewed, 12-18 month lag)
• Industry earnings calls (early signals, low credibility but high timeliness)

References

Peer-Reviewed and Authoritative Sources

1. Sevilla, J., & Roldán, E. (2024). "Training compute of frontier AI models grows by 4-5x per year." Epoch AI. May 28, 2024. https://epoch.ai/blog/training-compute-of-frontier-ai-models-grows-by-4-5x-per-year

   • Quality: A+ (authoritative research org, 14 years empirical data)

2. Cottier, B., Rahman, R., Fattorini, L., Maslej, N., & Owen, D. (2024). "The rising costs of training frontier AI models." arXiv:2405.21015v2. https://arxiv.org/abs/2405.21015

   • Quality: A (peer-reviewed preprint, 45 models analyzed 2016-2023)

3. Lu, C. (2025). "The Race to Efficiency: A New Perspective on AI Scaling Laws." arXiv:2501.02156. January 2025. https://arxiv.org/abs/2501.02156

   • Quality: B+ (arXiv preprint, theoretical framework, not yet peer-reviewed)
   • Key finding: Without ongoing efficiency gains, advanced performance could demand millennia of training or unrealistically large GPU fleets. Near-exponential progress remains achievable if efficiency-doubling rate parallels Moore's Law.

3b. "AI Scaling: From Up to Down and Out." arXiv:2502.01677v1. February 2025. https://arxiv.org/abs/2502.01677v1

• Quality: B+ (position paper, paradigm shift analysis, not yet peer-reviewed)
• Key finding: While Scaling Up faces inherent bottlenecks, future AI progress lies in Scaling Down (efficiency, reducing carbon footprint, equitable access) and Scaling Out (cross-domain collaboration). Emerging design pattern is agentic orchestration, not parameter-count escalation.

4. Epoch AI (2025). "Can AI Scaling Continue Through 2030?" Epoch AI Research. November 2025. https://epoch.ai/blog/can-ai-scaling-continue-through-2030

   • Quality: A (authoritative research org, forward projections)

5. "Revisiting Scaling Laws for Language Models." ACL 2025. aclanthology.org/2025.acl-long.1163.pdf

   • Quality: A- (peer-reviewed, limited to language modeling)

Industry Reports (Not Peer-Reviewed)

6. Bloomberg (November 13, 2024). "OpenAI, Google and Anthropic Are Struggling to Build More Advanced AI." https://www.bloomberg.com/news/articles/2024-11-13/openai-google-and-anthropic-are-struggling-to-build-more-advanced-ai

   • Quality: B+ (major outlet, multiple sources, but not peer-reviewed)

7. TechCrunch (December 21, 2024). "OpenAI's GPT-5 reportedly falling short of expectations." https://techcrunch.com/2024/12/21/openais-gpt-5-reportedly-falling-short-of-expectations/

   • Quality: B (tech journalism, corroborates Bloomberg)

8. TechCrunch (December 23, 2024). "OpenAI's o3 suggests AI models are scaling in new ways — but so are the costs." https://techcrunch.com/2024/12/23/openais-o3-suggests-ai-models-are-scaling-in-new-ways-but-so-are-the-costs/

   • Quality: B (industry coverage, test-time compute data)

9. The Decoder (2024). "Anthropic delays next-gen AI model Opus 3.5 with no new timeline." https://the-decoder.com/anthropic-delays-next-gen-ai-model-opus-3-5-with-no-new-timeline/

   • Quality: B- (tech journalism, CEO quote but strategic ambiguity)

Academic Preprints (Test-Time Compute)

10. "s1: Simple test-time scaling." arXiv:2501.19393. January 2025. https://arxiv.org/abs/2501.19393

    • Quality: B+ (arXiv preprint, reproduces o1 scaling curves)

11. "Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models." arXiv:2510.14232. October 2024. https://arxiv.org/abs/2510.14232

    • Quality: B+ (arXiv preprint, competitive programming benchmarks)

Summary

1. Correct the 3.6-month error: Sevilla & Roldán (2024) actually shows 5.9-month doubling, not 3.6 months
2. Acknowledge uncertainty: Wide confidence bands post-2024, honest about lack of peer-reviewed slowdown data

Key insight: Historical data supports faster scaling (5-6 months), late-2024 signals suggest slowdown (8-12+ months), but only historical period has peer-reviewed backing.