Analysis finds that AI model improvements in “reasoning” may slow down soon
The report’s findings suggest that progress from reasoning models could slow down as soon as within a year. As soon as within a year, progress from reasoning models could slow down, according to the report’s findings.
Reasoning models such as OpenAI’s o3 have led to substantial gains on AI benchmarks in recent months, particularly benchmarks measuring math and programming skills. The models can apply more computing to problems, which can improve their performance, with the downside being that they take longer than conventional models to complete tasks.
Reasoning models are developed by first training a conventional model on a massive amount of data, then applying a technique called reinforcement learning, which effectively gives the model “feedback” on its solutions to difficult problems. According to Epoch, so far, frontier AI laboratories like OpenAI don’t have a lot of computing power available to apply to the reinforcement-learning stage of training reasoning models.
That’s changing. OpenAI researcher Dan Roberts recently revealed that the company’s future plans call for prioritizing reinforcement learning to use far more computing power, even more than initial model training. And OpenAI researcher Dan Roberts recently revealed that the company’s future plans call for prioritizing reinforcement learning to use far more computing power, even more than for the initial model training.
But there’s still an upper bound to how much computing can be applied to reinforcement learning, per Epoch.
According to an Epoch AI analysis, reasoning model training scaling may slow down
Image Credits:
Epoch AIJosh You, an analyst at Epoch and the author of the analysis, explains that performance gains from standard AI model training are currently quadrupling every year, while performance gains from reinforcement learning are growing tenfold every 3-5 months. The progress of reasoning training will “probably converge with the overall frontier by 2026,” he continues.Epoch’s analysis makes a number of assumptions, and draws in part on public comments from AI company executives. It also argues that scaling reasoning models could be difficult for other reasons than computing, such as high overhead costs in research. You writes: “If research overhead costs are persistent, reasoning models may not scale as well as anticipated.”
It’s important to track this closely. “Rapid computation scaling could be a key ingredient in reasoning model development.” Studies have already shown that reasoning models are expensive to run and have flaws. For example, they tend to have more hallucinations than other models.
