The exact-search baseline should separate approximation loss from representation loss. In your example, both the indexed query and the forced exact scan operate on halfvec(1536), so the comparison measures ANN recall within the already quantized space. It does not show whether float16 storage changed the neighbors relative to the original float32 embeddings.
I would report two deltas: original-vector exact search versus halfvec exact search, then halfvec exact search versus its HNSW result. That would make the memory-saving decision independent of the index-tuning decision, particularly for nearly tied neighbors.
That’s a really good distinction. You’re right that comparing HNSW against an exact scan on halfvec only measures ANN recall within the quantized space, not the loss introduced by converting from float32 to float16. Separating those two deltas would make the trade-off much clearer, especially when nearest neighbors have very similar distances. Would you also recommend tracking ranking changes in addition to recall, since even small distance differences can affect the top-k ordering?
Ahmet Özel
AI Engineer. Computer Vision, RAG and LLM agents.
The exact-search baseline should separate approximation loss from representation loss. In your example, both the indexed query and the forced exact scan operate on halfvec(1536), so the comparison measures ANN recall within the already quantized space. It does not show whether float16 storage changed the neighbors relative to the original float32 embeddings.
I would report two deltas: original-vector exact search versus halfvec exact search, then halfvec exact search versus its HNSW result. That would make the memory-saving decision independent of the index-tuning decision, particularly for nearly tied neighbors.