Balancing Clusters To Reduce Response Time Variability In Large Scale Image Search

Tavenard Romain Inria - Irisa, Amsaleg Laurent Inria - Irisa, Jégou Hervé Inria - Irisa. Arxiv 2010

[Paper]    
ARXIV

Many algorithms for approximate nearest neighbor search in high-dimensional spaces partition the data into clusters. At query time, in order to avoid exhaustive search, an index selects the few (or a single) clusters nearest to the query point. Clusters are often produced by the well-known $k$-means approach since it has several desirable properties. On the downside, it tends to produce clusters having quite different cardinalities. Imbalanced clusters negatively impact both the variance and the expectation of query response times. This paper proposes to modify $k$-means centroids to produce clusters with more comparable sizes without sacrificing the desirable properties. Experiments with a large scale collection of image descriptors show that our algorithm significantly reduces the variance of response times without seriously impacting the search quality.

Similar Work

• Massively-parallel Heat Map Sorting And Applications To Explainable Clustering
• A Learned Index For Exact Similarity Search In Metric Spaces
• Joint Cluster Unary Loss For Efficient Cross-modal Hashing
• Exploiting Modern Hardware For High-dimensional Nearest Neighbor Search