How VPO Trains LLMs to Win at Test-Time Search with Ryan Bahlous-Boldi from MIT

in this tutorial we are going to take a deeper look at one of the overlooked levers in LLM post-training which is diversity, with Ryan Bahlous-Boldi from MIT! usually post-training with GRPO will straight up collapse the model onto a tight set of potential answers space (i.e. sharpening the distribution). however when you are running your LLM inside a search/discovery harness like alpha evolve where having multiple diverse candidate solution is an asset this sharpening is problematic. so Ryan and the team built Vector Policy Optimization (VPO), a drop-in replacement that trains the model to