Those blurry scan numbers made this click for me. 15% of the probability on the three legal values, and high still comes out 40% of the time looking exactly like a confident answer. In my extraction evals I'd also split the test set by input quality, since forced picks on bad scans can hide inside a healthy average the same way one bad schema does.
The 'unknown' enum is the underrated bit. I'd also log how often it fires per schema: a rate stuck at exactly zero on messy inputs suggests the model is still being cornered, not that the data is clean.
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
Those blurry scan numbers made this click for me. 15% of the probability on the three legal values, and high still comes out 40% of the time looking exactly like a confident answer. In my extraction evals I'd also split the test set by input quality, since forced picks on bad scans can hide inside a healthy average the same way one bad schema does.