An AI judge prefers answer A with probability 0.6. Should the preference model train on 0.6, or should we round it to a hard label of 1? Those choices optimize different objectives. The difference bec
Markdown supported · ⌘↵ to post
No responses yet.