With the rapid development of GPU (Graphics Processing Unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (Deep Neural Network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (Locality-sensitive Hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of datasets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.
翻译:随着图形处理器(GPU)技术与神经网络的快速发展,我们能够探索更适用的数据结构和算法。最新进展表明,神经网络可部分替代传统数据结构。本文提出了一种基于深度神经网络(DNN)的新型学习型局部敏感哈希方法LLSH,用于高效灵活地将高维数据映射至低维空间。LLSH采用并行多层神经网络替代传统LSH(局部敏感哈希)函数族,在保证查询精度的同时降低了时间和内存消耗。所提出的LLSH证明了用基于学习的神经网络替代哈希索引的可行性,为开发者更精确地设计和配置数据组织以提升信息搜索性能开辟了新途径。在不同类型数据集上的大量实验表明,该方法在查询精度、时间消耗和内存使用方面均具有显著优势。