🤖 Inference-Time Scaling for Generalist Reward Modeling
Failed to add items
Add to basket failed.
Add to wishlist failed.
Remove from wishlist failed.
Adding to library failed
Follow podcast failed
Unfollow podcast failed
🤖 Inference-Time Scaling for Generalist Reward Modeling
-
Narrated by:
-
By:
About this listen
This paper explores enhancing reward modeling (RM) for large language models (LLMs) by improving inference-time scalability. The authors introduce Self-Principled Critique Tuning (SPCT), a novel learning method that encourages RMs to generate their own guiding principles and accurate critiques through online reinforcement learning. Their approach, embodied in the DeepSeek-GRM models, utilizes pointwise generative reward modeling for greater flexibility. By employing parallel sampling and a meta RM to refine the reward voting process, they demonstrate significant improvements in the quality and scalability of their GRMs across various benchmarks. Notably, inference-time scaling with their method shows competitive or superior performance compared to simply increasing model size.