Reasoning models such as OpenAI’s o1 series and DeepSeek-R1 spend more compute at answer time — longer internal chains of thought, extra samples, sometimes a verifier — to raise accuracy on hard math, code, and science questions. That “think longer” bet is real. It is also more nuanced than a simple rule that more tokens always help. Lab work from 2025–26 shows that stretching a single trace can tip into overthinking, while parallel sampling plus majority vote often stretches the same budget further.
As an Amazon Associate I earn from qualifying purchases. If you buy through links on this page, I may earn a commission at no extra cost to you.
What “test-time compute” actually means
For a decade, the default story in large language models was train-time scaling: more data, more parameters, more pretraining FLOPs. Reasoning models add a second dial. At inference, the system may allocate a larger token budget before it commits to an answer — a long private chain of thought, multiple candidate solutions, search-like revision, or a separate check that the final answer is consistent.
OpenAI’s September 2024 introduction of o1-preview framed the idea plainly: models designed to spend more time thinking before they respond, with stronger results on science, coding, and math than prior chat models on the same hard evals. In the companion research note “Learning to reason with LLMs,” OpenAI reported that o1’s performance improved with both more reinforcement-learning train-time compute and more time spent thinking at test time — two different knobs, not one.
By December 2024, the production o1 API release added a practical product control: a reasoning_effort parameter so developers could trade latency and cost against thoroughness, and OpenAI said that snapshot used fewer reasoning tokens on average than o1-preview for a given request. The product lesson landed early: “think longer” is not only a research curve. It is a meter that customers eventually see on an invoice.
DeepSeek-R1 and the open reasoning wave
In January 2025, DeepSeek published DeepSeek-R1 (arXiv:2501.12948): a large-scale reinforcement-learning approach that incentivizes reasoning behaviors such as reflection, verification, and strategy shifts without depending on human-written chain-of-thought demos for the whole pipeline. The paper and GitHub release put weights and a detailed write-up in public view, which mattered for the research conversation even more than any single benchmark row.
DeepSeek’s materials contrast R1’s adaptive token use with fixed majority-vote or search recipes: fewer tokens on easy items, more on hard ones. They also note a familiar failure mode — overthinking on simpler questions, where the model keeps reasoning past the point of usefulness. That admission is useful. It matches what independent 2025 papers found when they tried to stretch one chain forever.
When longer traces stop helping
A tidy slogan — “more thinking equals better answers” — does not survive contact with careful measurement. Several 2025 studies looked at o1-like and R1-like models and asked whether extending a single thinking trace is the best use of a fixed inference budget.
One line of work, summarized in “Does Thinking More Always Help? Mirage of Test-Time Scaling in Reasoning Models” (arXiv:2506.04210), reports a non-monotonic pattern: accuracy can rise with extra thinking steps, then plateau or fall as models revise, second-guess, or abandon an already-correct path. The authors argue that some apparent gains from “wait, rethink” prompting are a mirage tied to uncertainty and how eval metrics behave, not a clean proof that sequential stretch is the right scaling law.
A related February 2025 study, “Revisiting the Test-Time Scaling of o1-like Models” (arXiv:2502.12215), compared sequential revision with parallel sampling under similar token budgets. Parallel coverage improved faster; sequential revision helped only up to a point and carried higher attention cost over long contexts. Their “Shortest Majority Vote” variant preferred clusters that were both popular and short — a practical hint that the first clean answer is often better than the longest one.
“Don’t Overthink it” (arXiv:2505.17813) pushes the same theme harder: within a question, shorter sampled chains were substantially more likely to be correct than the longest chain for that same question. Their short-m@k method runs several generations in parallel, stops when the first m finish, and votes among those shorter completions — aiming for majority-vote quality with less wall-clock time and fewer thinking tokens.
None of this says reasoning models are a fad. It says the shape of the compute matters. Serial “keep talking” is not the same as “sample a few independent attempts and pick the consensus.”
Parallel sampling, majority vote, and verifiers
Classic test-time recipes still matter: sample N answers, take the majority; or sample N and let a reward model or verifier pick a winner (best-of-N). Those ideas predate o1, but reasoning models made them newly expensive — and newly valuable — because each sample can be a long, costly thinking trace.
The 2025 “mirage” paper’s parallel-thinking alternative is straightforward: split the same token budget across independent paths and vote. In their controlled comparisons, that beat stretching one path with “wait / think more” style prompts by a wide margin under a fixed budget. You do not need to treat the exact percentage lifts as universal laws; the directional lesson is enough for product teams. Diversity of attempts often buys more than monologue length.
Verifiers add another layer when the domain allows a check — unit tests for code, a symbolic checker for math, or a smaller judge model. OpenAI’s own January 2025 note on trading inference-time compute for adversarial robustness also frames longer thinking as a resource that can improve behavior under attack in some settings, while acknowledging cases where more compute does not help. Again: budget is a tool, not a magic wand.
Cost, latency, and what products should actually ship
For builders, the interesting question is not “does test-time compute work?” It is “when is the bill worth it?”
- Hard, verifiable tasks. Contest math, tricky coding, multi-step scientific questions — domains where a wrong answer is obvious after the fact — are where extra inference compute has repeatedly shown up in vendor evals and open research.
- Easy or fuzzy tasks. Short factual lookups, tone edits, and open-ended brainstorming often do not need a marathon chain of thought. Overthinking here burns latency and money for little gain.
- Interactive products. Users feel seconds. A reasoning_effort-style control, a cheap first pass with a “think harder” button, or an async “deep solve” job is usually kinder than making every chat turn wait for a long private monologue.
- Batch and agent workloads. Overnight evals, code agents with sandboxes, and research pipelines can afford more parallel samples. Majority vote and short-m@k style early-stop tricks fit those settings better than a single ultra-long trace.
There is also an infrastructure angle. Training still dominates the biggest capital spends, but inference is where perpetual operating cost lives. If every hard query multiplies tokens by ten or a hundred, capacity planning and pricing have to change — even when accuracy charts look great in a paper.
The bottom line
Test-time compute is a real second axis of progress for language models. OpenAI’s o1 work and DeepSeek’s R1 release made “spend more to think” a mainstream product and research story. The follow-up science is just as important: longer is not automatically smarter. Parallel attempts, majority vote, shorter preferred chains, and domain verifiers often stretch a fixed budget better than one endless revision spiral.
If you are shipping a reasoning feature in 2026, treat thinking budget like any other scarce resource. Put the heavy dial on hard, checkable work. Keep a fast path for everything else. And measure accuracy against wall-clock time and dollars — not against how impressive the hidden chain of thought looks in a demo.
Further reading
Thinking, Fast and Slow — Daniel Kahneman’s classic on two modes of thinking — a readable backdrop for why “more deliberation” sometimes helps and sometimes backfires.
A Brief History of Intelligence — Max Bennett’s tour of brain evolution and AI breakthroughs — useful context for today’s bet on machines that spend more compute to reason.