Every Greedy Metric Said the Model Was Improving. Then pass@64 Fell From 0.83 to 0.19
DEV Community
Every Greedy Metric Said the Model Was Improving. Then pass@64 Fell From 0.83 to 0.19
Outcome-only RL on a toy system looked great on greedy metrics while silently destroying the sampling channel that search-based deployment relies on. A registered-prior falsification story.
0 comments
No comments yet.