This technical report describes a novel approach to improving the reasoning capabilities of large language models (LLMs) by employing a reward-guided tree search framework. The framework consists of three key components: a policy model to generate reasoning steps, a reward model to provide feedback, and a search algorithm to guide the exploration of potential solutions. The authors explore various design considerations for each component and evaluate their approach on several challenging mathematical datasets, demonstrating significant improvements in reasoning abilities.

https://arxiv.org/pdf/2411.11694