A minimal perfect hash function (MPHF) maps a set of n keys to the first n integers without collisions. Representing this bijection needs at least $\log_2(e) \approx 1.443$ bits per key, and there is a wide range of practical implementations achieving about 2 bits per key. Minimal perfect hashing is a key ingredient in many compact data structures such as updatable retrieval data structures and approximate membership data structures. A simple implementation reaching the space lower bound is to sample random hash functions using brute-force, which needs about $e^n \approx 2.718^n$ tries in expectation. ShockHash recently reduced that to about $(e/2)^n \approx 1.359^n$ tries in expectation by sampling random graphs. With bipartite ShockHash, we now sample random bipartite graphs. In this paper, we describe the general algorithmic ideas of bipartite ShockHash and give an experimental evaluation. The key insight is that we can try all combinations of two hash functions, each mapping into one half of the output range. This reduces the number of sampled hash functions to only about $(\sqrt{e/2})^n \approx 1.166^n$ in expectation. In itself, this does not reduce the asymptotic running time much because all combinations still need to be tested. However, by filtering the candidates before combining them, we can reduce this to less than $1.175^n$ combinations in expectation. Our implementation of bipartite ShockHash is up to 3 orders of magnitude faster than original ShockHash. Inside the RecSplit framework, bipartite ShockHash-RS enables significantly larger base cases, leading to a construction that is, depending on the allotted space budget, up to 20 times faster. In our most extreme configuration, ShockHash-RS can build an MPHF for 10 million keys with 1.489 bits per key (within 3.3% of the lower bound) in about half an hour, pushing the limits of what is possible.
翻译:最小完美哈希函数(MPHF)将n个键的集合无冲突地映射到前n个整数。表示这种双射每个键至少需要$\log_2(e) \approx 1.443$比特,现有多种实用实现可达约2比特/键。最小完美哈希是许多紧凑数据结构的核心组件,如可更新检索数据结构和近似成员查询数据结构。达到空间下界的简单实现是暴力采样随机哈希函数,期望尝试次数约$e^n \approx 2.718^n$次。近期提出的ShockHash通过采样随机图将期望尝试次数降至约$(e/2)^n \approx 1.359^n$。本文提出的二分ShockHash则采样随机二分图。我们描述二分ShockHash的通用算法思想并进行实验评估。关键创新在于:尝试两种哈希函数的所有组合,每个函数映射至输出范围的一半。这使得采样哈希函数的期望数量降至仅约$(\sqrt{e/2})^n \approx 1.166^n$。但该优化本身未显著降低渐进运行时间,因为仍需测试所有组合。然而,通过组合前对候选函数进行过滤,可进一步将期望组合数降至低于$1.175^n$。实验表明,二分ShockHash的实现速度较原始ShockHash提升高达三个数量级。在RecSplit框架内,二分ShockHash-RS支持显著更大的基例,使得构建速度(根据分配的空间预算不同)最高提升20倍。在最极端配置下,ShockHash-RS可在约半小时内为1000万键构建MPHF,实现每键1.489比特(距理论下界仅3.3%),突破现有极限。