Comment by Kartik N V J K on "A hex literal made LLM code reviewers raise false alarms"

Kartik N V J K

AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor

With a 3072-bit key written as 0xC00 getting flagged, I'd want to read the reviewer's stated reason next to the flag. If it says the key is too small, it misread the value. If it complains about a magic number, a style note got scored as a security finding. Did the false positives come with reasons you could sort that way?

SC

Sunny Chokshi

Doctoral student researching LLM security

6d ago

Yes, I read the replies for these. For the two 3072-bit twins (0xC00 and 3 * 2 ** 10), 24 replies flagged them. All 24 decoded the value correctly as 3072. 21 then called it too small, weak or insufficient, two called it non-standard, and one cited the backend. So the models read the value and still judged it wrongly. Neither of your two cases fits exactly. It is a wrong stated reason, not a misread number. Sorting every false positive this way is the check I would add next.

Search Hashnode