A Monotone Minimal Perfect Hash Function (MMPHF) constructed on a set S of keys is a function that maps each key in S to its rank. On keys not in S, the function returns an arbitrary value. Applications range from databases, search engines, data encryption, to pattern-matching algorithms. In this paper, we describe LeMonHash, a new technique for constructing MMPHFs for integers. The core idea of LeMonHash is surprisingly simple and effective: we learn a monotone mapping from keys to their rank via an error-bounded piecewise linear model (the PGM-index), and then we solve the collisions that might arise among keys mapping to the same rank estimate by associating small integers with them in a retrieval data structure (BuRR). On synthetic random datasets, LeMonHash needs 35% less space than the next best competitor, while achieving about 16 times faster queries. On real-world datasets, the space usage is very close to or much better than the best competitors, while achieving up to 19 times faster queries than the next larger competitor. As far as the construction of LeMonHash is concerned, we get an improvement by a factor of up to 2, compared to the competitor with the next best space usage. We also investigate the case of keys being variable-length strings, introducing the so-called LeMonHash-VL: it needs space within 10% of the best competitors while achieving up to 3 times faster queries.
翻译:针对密钥集合S构建的单调最小完美哈希函数(MMPHF)是一种将S中每个密钥映射到其秩的函数。对于不在S中的密钥,该函数返回任意值。其应用涵盖数据库、搜索引擎、数据加密及模式匹配算法等领域。本文描述了LeMonHash——一种为整数构建MMPHF的新技术。LeMonHash的核心思想出奇简单且高效:通过误差有界分段线性模型(PGM索引)学习从密钥到其秩的单调映射,然后通过关联检索数据结构(BuRR)中的小整数,来解决可能出现在映射到相同秩估计值密钥间的冲突。在合成随机数据集上,LeMonHash比次优竞争者节省35%的空间,同时查询速度提升约16倍。在真实世界数据集上,其空间使用量接近或显著优于最优竞争者,且查询速度比次优竞争者最高提升19倍。就LeMonHash的构建而言,相比空间使用量次优的竞争者,我们获得了最高2倍的性能提升。我们还研究了密钥为可变长度字符串的情况,提出了名为LeMonHash-VL的方案:其在保持空间占用与最优竞争者差距在10%以内的同时,实现了最高3倍的查询加速。