Gaussian Kernel-Based LSH for High-Dimensional Similarity Search

Abstract

High-dimensional similarity search remains a critical challenge in machine learning, particularly when data lie on complex, non-linear manifolds that undermine the effectiveness of classical Locality-Sensitive Hashing (LSH). This work introduces Gaussian LSH, a kernel-based hashing framework that integrates over-clustering with Gaussian probability density modelling to improve locality preservation while maintaining computational efficiency. The method generates compact binary codes from a hybrid kernel–PDF score and supports scalable GPU-accelerated indexing for large datasets. Empirical evaluations across multiple visual and textual benchmarks demonstrate consistent improvements in recall and query latency compared to representative LSH variants and approximate nearest neighbour libraries. Gaussian LSH achieves recall gains of up to $\text{9}\,\text{pp}$ and latency reductions of up to $4.3\times$, with benefits sustained across a range of code lengths. These results highlight the approach’s scalability and accuracy, supporting its use in medium- to large-scale similarity retrieval tasks across diverse data domains.