Search-time scaling refers to the empirical finding that increasing the amount of search performed at inference time — rather than increasing model parameters or training data — reliably improves task performance. It is the inference-side analog of the training-time scaling laws documented by Kaplan et al., and it suggests that compute budgets can be traded between training and serving depending on the application's cost constraints.

Think of it like adjusting the depth of analysis a consultant performs per engagement. A smaller firm spending more hours per client can match the output quality of a larger firm that spends fewer hours, up to a point determined by the consultant's underlying knowledge.

Empirical evidence for search-time scaling comes from domains including mathematical reasoning and code generation. At a fixed model size, generating 256 candidate solutions and selecting the best using a verifier outperforms greedy decoding by a margin that grows with the number of candidates. The curve typically follows a log-linear relationship: each doubling of inference compute yields a roughly constant improvement in accuracy on hard benchmarks.

Search-time scaling interacts directly with Inference-Time Search, which provides the mechanism, and Process Reward Models, which provide the scoring function. Test-Time Compute is the resource being scaled. The phenomenon is relevant to any LLM deployed via API because it implies that a smaller, cheaper Foundation Model with more inference compute per query may be preferable to a larger model for certain workloads. Chain of Thought prompting amplifies search-time gains by structuring the search space. Mixture of Experts architectures change the compute profile of each forward pass, which affects how search-time budgets should be allocated.

You will encounter this concept when designing serving infrastructure, comparing model tiers for cost-sensitive applications, or reading scaling-law research papers.