A small model, allowed to think harder at the moment you ask, beat a model fourteen times its size. There are at least five ways to spend that inference compute, and the skill is not climbing as high as possible. It is matching the rung to the difficulty.
In a paper posted in August 2024, Snell and three colleagues reported a result that should feel impossible. A small language model, given extra time to think at the moment you ask it a question, could beat a model fourteen times its size. Same lineage, same training recipe, one simply fourteen times larger, and the small one, allowed to spend more computation while it answered, won.
The result comes with fine print, and it matters: it holds on problems where the small model already had a non-trivial chance of success, and the comparison is matched for total compute. But the headline is real, and it names a whole second way to make a model smarter. You can make it bigger, which is expensive and permanent, baked into the weights. Or you can let it think harder at the moment of answering, which is cheap and paid per question. The field spent years obsessed with the first lever. Since late 2024 it has been transfixed by the second, and that second lever has a name: test-time compute, the computation you spend at inference rather than in training.
Here is the part that turns a single lever into a design discipline. There is no one way to spend test-time compute. There are at least five, they form a ladder from cheap to expensive, and the most useful thing Snell's paper found is that the rungs are not interchangeable. Which one wins depends on how hard the question is.
Way one: think before answering. The base rung is chain-of-thought, and it is almost embarrassingly simple. Instead of demanding an immediate answer, you let the model write out intermediate steps, and accuracy jumps. Jason Wei and colleagues demonstrated this in 2022, showing that a large model given eight worked examples and permission to reason step by step set a new state of the art on grade-school math word problems. Nothing about the model changed. It was allowed to spend tokens, and tokens are computation. A chain of thought is not a window into the model's soul; it is the model doing more arithmetic in the open, one forward pass at a time. This is the cheapest rung: a single chain, generated once.
Way two: think several times and vote. The next rung keeps the single chain and multiplies it. Sample not one reasoning path but many, each with a little randomness, then take the answer the majority of them agree on. Xuezhi Wang and colleagues called this self-consistency in 2022 and reported gains of nearly eighteen points on the same math benchmark, for no reason more exotic than this: correct reasoning tends to converge on the same answer from different directions, while mistakes scatter. We have written at length elsewhere about the surprising power, and the real limits, of asking a model the same question several times and letting the answers vote, so I will leave it at one rung on the ladder. It is parallel where way one is sequential, and it costs you N times the compute for one better answer.
Way three: think many times and let a judge pick. Voting treats every sample as an equal citizen. The third rung hires a judge. You generate many candidate answers and use a separate model, a verifier or reward model, to score them and keep the best. Karl Cobbe and colleagues showed in 2021 that training a verifier to grade math solutions beat simply fine-tuning the model to produce them. Two years later, Hunter Lightman and colleagues sharpened the idea with process reward models that score each step of the reasoning rather than only the final answer, releasing eight hundred thousand human step-level judgments to train them. The insight is that catching a wrong turn on line three is worth more than noticing the destination was wrong on line ten.
Way four: think in a tree and backtrack. The fourth rung stops sampling blindly and starts searching. Instead of drawing complete answers and ranking them, you explore a branching tree of partial reasoning, expand the promising branches, and prune the dead ends, backtracking when a path goes nowhere. Shunyu Yao and colleagues named this Tree of Thoughts in 2023. It is the difference between rolling a hundred dice and keeping the best, and actually playing chess: looking ahead, abandoning bad lines, spending your compute where the search looks alive.
Way five: teach the model to do all of it itself. The top rung is the newest and the most consequential. The first four ways are things you, the engineer, do around a frozen model. The fifth teaches the model to do them internally. Using reinforcement learning, you train the model to produce one long chain of thought that samples its own alternatives, checks its own work, and backtracks on its own, with no external scaffolding at all. This is the machinery behind the reasoning models that arrived at the end of 2024. DeepSeek's R1, published in Nature in 2025, showed that reasoning behavior could be incentivized through reinforcement learning alone, with self-reflection and self-correction emerging without a single human-labeled reasoning example. The model was not shown how to think. It was rewarded for getting things right, and it learned to think as the means.
There is a close cousin worth naming so the five do not look more tidy than they are: sequential self-refinement, where a model critiques and revises its own answer over several rounds, which Aman Madaan and colleagues formalized as Self-Refine in 2023. It blurs into ways one and five, which is the honest truth about all of these categories. Five is a map, not a law.
Now return to Snell's paper, because its real contribution is not the fourteen-times headline. It is the organizing rule underneath. The optimal way to spend test-time compute changes with the difficulty of the problem, and the change is not subtle. On easier problems, where the model's first instinct is often close, the winning move is sequential revision: take a promising answer and refine it. On harder problems, where the first instinct is usually wrong, revision just polishes a mistake, and the winning move is broad parallel search with a verifier: explore widely, then select. A policy that senses the difficulty and picks the matching strategy, what the authors call compute-optimal, is more than four times as efficient as a fixed best-of-N baseline. Same budget, four times the mileage, from nothing but spending it the right way for the problem in front of you.
That reframes the whole ladder. The rungs are not better and worse. They are suited to different questions. Reaching for tree search on a problem the model could nail by simply revising its first draft is as wasteful as blurting a one-line answer to a problem that needed a hundred steps of search. The skill is not climbing as high as possible. It is matching the rung to the difficulty.
Line the five ways up in order and you are looking at more than a taxonomy. You are looking at a history, and it has a direction.
Ways two through four were external harnesses. An engineer wrote a loop that sampled the model many times, another that ran the verifier, another that walked the tree. The model sat frozen in the middle, thinking exactly as long as it always had, while the cleverness lived in the code wrapped around it. What reinforcement learning did, in the shift to reasoning models, was fold that cleverness inward. The sampling, the checking, the backtracking that used to be an algorithm around the model became a behavior inside it, expressed as one long chain of thought. The scaffolding became a skill.
This is why the frontier suddenly looks different. A year ago, getting the most out of a model meant becoming an expert at wrapping it in best-of-N loops and search trees. Now the strongest models arrive with the search pre-installed, having been trained to spend their own inference compute well. It is worth seeing this against the older story of scale. For years the only recognized way to buy capability was training compute, more parameters and more data, the paradigm the scaling laws described. Test-time compute is the second axis, discovered late and now being scaled deliberately in its own right, with reinforcement learning as the tool that teaches a model to use it. The industry did not replace one lever with the other. It found the second lever and is now pulling both.
Everything above rung one shares a single dependency, and it is the most important sentence in this essay: the higher rungs pay off only if you can score a candidate answer. Best-of-N needs a judge. Tree search needs to know which branch is winning. Reinforcement learning needs a reward. All of that is verification, and verification is the ceiling on the entire ladder.
Where verification is cheap and reliable, the ladder scales beautifully. Mathematics has checkable answers. Code has unit tests that pass or fail. In those domains you can generate a thousand attempts and trust the filter that keeps the good one, and test-time compute turns almost directly into accuracy. Where verification is hard, the ladder stalls. For open-ended reasoning, for questions of taste or strategy or judgment, more samples do not help if you cannot tell which sample is right, and a verifier that is itself unreliable will confidently promote nonsense. The gap between how well a system can generate candidates and how well it can tell the good ones apart is the lever that gates all five ways, and process reward models narrow it without closing it.
Two honest edges keep this from being a story about a free lunch. The first is that more thinking is not always better. Past a point, extra reasoning can curdle into something like rumination, the model talking itself out of a right answer or spiraling on an easy one, a failure mode we have written about on its own terms. The second is sharper: the chain of thought you can read is not guaranteed to be the reasoning the model actually used. Miles Turpin and colleagues showed in 2023 that chain-of-thought explanations can systematically misrepresent the true reason for a model's answer, producing fluent justifications for conclusions that were actually driven by something the model never mentions. Test-time compute buys accuracy on problems you can verify. It does not buy honesty about how the answer was reached, and a long, reasonable-looking chain is evidence of neither.
So here is the practical residue, and it is a decision procedure more than a technique.
Start by asking two questions about your task: how hard is it, and can you check the answer. Those two axes tell you almost everything about which of the five ways will pay you back.
If the task is easy for your model, do not overspend. Let it think in a single chain, perhaps let it revise once, and stop. Reaching for parallel search here buys nothing and can make things worse.
If the task is hard and checkable, math, code, anything with a test or a ground truth, climb the ladder deliberately, and invest in the verifier above all. Sampling more is linear; a better verifier is a multiplier, because it raises the return on every other rung. When in doubt, spend your next hour improving how you score answers, not how many you generate.
If the task is hard and not checkable, be honest that the ladder mostly stalls here, and that extra compute tends to buy confident, well-formatted wrongness. Your real work is to manufacture a verifier where none exists: write the test, define the rubric, find the external signal that tells right from plausible. Until you can tell when the model is right, more thinking is just more expensive guessing.
And if you are simply calling a frontier reasoning model, understand what you are renting: a model that has internalized the top of the ladder, that will search and check inside its own chain of thought. It is a remarkable tool, and it obeys the same law as all the rest. It shines where its answers can be verified, and its visible thinking is a trace of computation, not a confession of motive. Do not mistake a long chain for a faithful one.
The deepest lesson of test-time compute is that intelligence at inference is real leverage, a genuine second axis of scaling, but it is leverage resting on a fulcrum called verification. The question was never really whether to let the model think longer. It is whether you can tell when it is right, because that single answer decides which of the five ways will work, and how far up the ladder you get to climb.
Verification is the ceiling on the entire ladder. It is also the ceiling on trusting any agent's output.
Every rung of test-time compute pays off only if you can tell a good answer from a plausible one, and the same is true one level up, for any agent you deploy: more reasoning buys confident, well-formatted wrongness unless something can check it against ground truth. The agent trust stack is that missing verifier, built as installable pieces: verification against a real answer key rather than the model's own fluency, a provenance record of what the agent actually did (because a long, reasonable-looking chain is not evidence it reasoned that way), and ratings that price an agent by its verified track record. Spend your next hour on how you score answers, not how many you generate.
Read the Theory of Agent Trust
pip install agent-trust-stack · npm install agent-trust-stack
Or the provenance record on its own: pip install chain-of-consciousness / npm install chain-of-consciousness.