Tree of Thought (ToT) is a prompting framework that structures model reasoning as a branching search tree rather than a single linear chain of steps. At each decision point the model generates multiple candidate thoughts, evaluates them, and selectively expands the most promising ones — repeating this process until a solution is reached or a depth limit is hit. This makes it particularly effective for problems that require exploration and backtracking rather than straightforward step-by-step derivation.

Think of it like a chess player mentally running through different moves and counter-moves before committing to one. Rather than simply reacting to the current board state, the player weighs branches of possibility, prunes losing lines, and deepens the search along lines that look strongest. ToT gives a language model that same structured exploration instead of forcing it to commit to a single path immediately.

A practical use looks like this: given the puzzle "what five-letter word becomes a new word when you remove each letter in turn?", a ToT prompt first generates a handful of candidate words, then asks the model to verify each candidate by simulating the removal steps, then discards failures and continues only the surviving branches.

Step 1 – Generate candidates: ["crane", "stare", "plane"]
Step 2 – Evaluate "crane": c-r-a-n-e → rane/cane/crne/crae/cran – partial match, keep
Step 3 – Backtrack and try next candidate if evaluation fails

This structured back-and-forth is something a flat Chain of Thought prompt cannot do alone.

ToT connects closely to Chain of Thought, Prompt Engineering, and LLM inference strategies; understanding Temperature helps because diverse candidate generation usually benefits from a slightly higher setting. Zero-Shot and few-shot approaches are the simpler baselines it improves upon.

You will encounter Tree of Thought when tackling math olympiad problems, multi-step planning tasks, or puzzle-solving benchmarks where models powered by a single reasoning chain consistently fall short.