Distance
Euclidean typical. Cosine for high-dim (text embeddings). Manhattan for sparse.
Advertisement
Complexity
Query O(N·d) naive. k-d tree O(log N) low-d. LSH/HNSW O(log N) high-d.
Advertisement
Curse of dimensionality
Above ~10-20 dims, all points nearly equidistant. Distance-based methods degrade. Use approximate methods.
Weighted variants
Weight neighbors by 1/distance. Locally-weighted regression: fit local model per query. LOESS.