A Process Reward Model (PRM) is a neural network trained to assign a scalar reward to each intermediate step in a reasoning chain rather than only to the final answer. This contrasts with an outcome reward model, which scores only the end result. By providing dense, per-step feedback, PRMs guide search algorithms toward correct reasoning paths even when the final answer would otherwise look identical for correct and flawed derivations.

Think of it like a driving instructor who gives feedback at each turn — not just a thumbs-up or thumbs-down when you reach the destination. Knowing that you took a wrong route midway lets you correct course before the mistake compounds.

PRMs are particularly valuable for mathematical and logical reasoning tasks. A model generating multiple chain-of-thought candidates produces step-by-step traces, and the PRM scores each step. Inference-time search can then use these scores to prune incorrect paths early. OpenAI published research in 2023 showing that process supervision significantly outperformed outcome supervision on the MATH benchmark when combined with beam search.

PRMs are trained using RLHF techniques, where human annotators or automated verifiers label individual reasoning steps as correct or incorrect. They work alongside Foundation Models and complement Chain of Thought prompting by turning qualitative reasoning guidance into quantitative scores. The PRM's output feeds directly into Test-Time Compute budgets: steps with low scores trigger backtracking. Fine-tuning a PRM on domain-specific problems can specialise it far beyond its general training, similar to how LoRA adapts a base model.

You will encounter this term when reading about reasoning-optimised models, math problem-solving benchmarks, or inference-time search infrastructure.