Second-Guessing Has a Price

A poker player holding a decent hand faces a genuine decision each round. Calling costs chips now for a chance at a bigger pot later, and folding gives up whatever is already in play to avoid a loss that has not happened yet. Neither choice is obviously correct, and the right one depends on what is already known about the hand, not on how long the player stares at the cards. A team at the University of Science and Technology of China built something close to that same instinct into large reasoning models, the class of language models trained to generate long chains of intermediate reasoning steps before producing a final answer. Extending that reasoning costs compute, and the researchers argue the decision to keep going should depend on whether more thinking is likely to pay off, not on a fixed token budget handed down in advance ^[2].

Test-time compute, the amount of reasoning a model generates while answering a single query, has become one of the main levers for improving accuracy on hard problems, alongside training on more data or building bigger models. The general finding, replicated across many benchmarks, is that letting a model reason at length before answering tends to raise accuracy on tasks like competition mathematics. What has drawn less scrutiny is whether that relationship holds all the way up, or whether it eventually bends.

When the Curve Bends the Wrong Way

A team at Nanjing University set out to measure exactly that. Working with reasoning models forced to keep generating for a controlled number of tokens ranging from 500 up to 16,000, using a technique that appends the word Wait if the model tries to stop early, the researchers tracked how each additional block of reasoning tokens changed accuracy ^[1]. Borrowing a concept from economics, the law of diminishing marginal returns, where each additional unit of input produces a smaller gain in output than the one before it, they found that the marginal benefit of extra reasoning tokens shrinks steadily as the budget grows, and for easier problems it can turn negative well before the budget runs out. Generating 8,000 tokens costs roughly sixteen times what 500 tokens costs, and much of that spending on simple problems buys nothing ^[1].

More strikingly, the researchers tracked individual problems through their reasoning trajectories and identified what they call flip events, moments where a model arrives at the correct answer relatively early, then keeps reasoning anyway and talks itself into a wrong one. Extended thinking was not simply wasted in these cases, it was actively harmful, with the model second-guessing an initial answer that had already been right ^[1]. The severity of this pattern varied by model and by problem difficulty, easier problems flipped from correct to incorrect earlier in the reasoning trace than harder ones, which suggests that a single fixed thinking budget applied uniformly across every query is close to the worst way to spend a compute budget.

Pouring Past Full

Water poured into a glass that is already full does not raise the water level. It spills over the rim, and if the glass gets jostled in the process, some of what was already sitting there can end up on the table too. A model that keeps generating reasoning tokens well after it has settled on an answer is pouring past that same rim, spending compute that buys nothing, and every so often jostling loose a correct answer that would have stood if the pour had simply stopped.

Betting Only When the Odds Are Good

The USTC team's system, called Bet for budget-efficient thinking, tries to learn that same read on the table, when continued reasoning is worth its cost and when it is not. Rather than sizing the compute budget to how hard a problem looks on its surface, which the researchers point out is a poor proxy, Bet estimates how likely the model's current policy is to actually solve the problem if given more time, a quantity closer to solvability than to difficulty ^[2]. A problem can look intimidating and still be within easy reach for a well-trained model, and a problem that reads simply can sit just outside what the model can currently do no matter how long it reasons. Based on this estimate, the system learns three behaviors described in the paper's own poker terms, a short solve that answers easy queries concisely, a nice fold that abstains early once continued reasoning shows near-zero expected return, and a hero call that commits substantial compute to problems that are hard but genuinely within reach. Trained with a two-stage process combining supervised examples with reinforcement learning under a cost-aware reward, Bet reportedly cut reasoning tokens by around 55 percent on average across seven benchmarks while improving overall accuracy, and the learned behavior transferred to scientific and logical reasoning tasks outside the mathematics domain it was trained on ^[2].

Not Every Turn Deserves the Same Budget

Most budget-allocation research, including the two studies above, treats a query as a single, self-contained reasoning problem. A team at Carnegie Mellon University points out that this framing breaks down once a model is having a multi-turn conversation, since a fixed per-turn budget spent generously on an easy opening question leaves less room for a much harder follow-up question three turns later ^[3]. Their system, called TAB for turn-adaptive budgets, treats the whole conversation as a sequential compute allocation problem, formally a multi-objective decision process where the model has to decide, turn by turn, how much of a shared token budget to spend now versus hold in reserve for whatever comes later in the exchange. Trained with a group-based reinforcement learning method, TAB was reported to save up to 35 percent of tokens compared with static, turn-unaware budget policies while maintaining comparable accuracy on mathematical reasoning benchmarks, and a variant given advance knowledge of all the sub-questions in a conversation saved up to 40 percent ^[3].

Pacing Across an Uneven Track

A hurdles race is not run at one constant effort. The open stretches between barriers call for a different pace than the barrier itself, and a runner who sprints flat out over every meter arrives at the tenth hurdle with nothing left to clear it. TAB spends its tokens the way that runner spends stride length, freely across the open stretches of a conversation, held back for whichever turn turns out to be the barrier.

Teaching the Habit During Training, Not Just at Inference

A fourth angle on the same problem moves the intervention earlier, into training itself. A framework called BACR, for budget-adaptive curriculum reasoning, argues that training a model under a single fixed token budget, or under budgets sampled uniformly at random, teaches it a compute habit that does not match the actual spread of problem difficulty it will face later ^[4]. BACR conditions the model on its assigned budget as an explicit input during training, then uses a scheduler that shifts the distribution of training budgets from easy to hard as the model's own performance improves, paired with a reward that gives partial credit for intermediate reasoning steps even when a response gets cut off by the budget. On a set of mathematics benchmarks including MATH and AIME, the reported result was up to an 8.3 percent accuracy improvement under tight budgets alongside a 34 percent reduction in average token consumption compared with training under unconstrained reasoning ^[4]. The gain under tight budgets specifically is the detail worth sitting with, since a curriculum that only helps when compute is already generous would be a much smaller contribution.

What This Suggests

None of these four systems agree on exactly where the compute-allocation decision should live. Bet puts it inside a single query, TAB puts it across the turns of a conversation, and BACR puts it back in training before any of that decision-making happens at inference time. Taken together they point toward the same underlying claim, that a uniform thinking budget is a specific and avoidable form of waste rather than a neutral default, and that the waste sometimes curdles into an actual accuracy cost through the flip events the Nanjing University team documented. Every reported gain here comes from mathematical reasoning benchmarks or benchmarks adjacent to them, and it remains an open question whether solvability estimates, turn-level budgeting, and curriculum scheduling generalize as cleanly to open-ended tasks like coding or long document analysis, where correctness is harder to check and a flip event is harder to even detect. From a systems perspective, the more durable lesson may be procedural rather than architectural, that any team deploying test-time compute at scale should be measuring the marginal accuracy per token spent, not just the accuracy at whatever budget happens to be the current default.