In this work we study Invertible Bloom Lookup Tables (IBLTs) with small failure probabilities. IBLTs are highly versatile data structures that have found applications in set reconciliation protocols, error-correcting codes, and even the design of advanced cryptographic primitives. For storing $n$ elements and ensuring correctness with probability at least $1 - \delta$, existing IBLT constructions require $\Omega(n(\frac{\log(1/\delta)}{\log(n)}+1))$ space and they crucially rely on fully random hash functions. We present new constructions of IBLTs that are simultaneously more space efficient and require less randomness. For storing $n$ elements with a failure probability of at most $\delta$, our data structure only requires $\mathcal{O}\left(n + \log(1/\delta)\log\log(1/\delta)\right)$ space and $\mathcal{O}\left(\log(\log(n)/\delta)\right)$-wise independent hash functions. As a key technical ingredient we show that hashing $n$ keys with any $k$-wise independent hash function $h:U \to [Cn]$ for some sufficiently large constant $C$ guarantees with probability $1 - 2^{-\Omega(k)}$ that at least $n/2$ keys will have a unique hash value. Proving this is non-trivial as $k$ approaches $n$. We believe that the techniques used to prove this statement may be of independent interest. We apply our new IBLTs to the encrypted compression problem, recently studied by Fleischhacker, Larsen, Simkin (Eurocrypt 2023). We extend their approach to work for a more general class of encryption schemes and using our new IBLT we achieve an asymptotically better compression rate.

翻译：在本研究中，我们探讨了具有低失败概率的可逆布鲁姆查找表。IBLT 是一种高度通用的数据结构，已在集合协调协议、纠错码乃至高级密码原语的设计中得到应用。对于存储 $n$ 个元素并确保正确率至少为 $1 - \delta$ 的场景，现有的 IBLT 构造需要 $\Omega(n(\frac{\log(1/\delta)}{\log(n)}+1))$ 的空间，并且严重依赖于完全随机的哈希函数。我们提出了新的 IBLT 构造，在提升空间效率的同时降低了对随机性的需求。对于存储 $n$ 个元素且失败概率至多为 $\delta$ 的情况，我们的数据结构仅需 $\mathcal{O}\left(n + \log(1/\delta)\log\log(1/\delta)\right)$ 的空间和 $\mathcal{O}\left(\log(\log(n)/\delta)\right)$-wise 独立哈希函数。作为一个关键的技术要素，我们证明了对于任意 $k$-wise 独立哈希函数 $h:U \to [Cn]$（其中 $C$ 为足够大的常数），哈希 $n$ 个键时，以 $1 - 2^{-\Omega(k)}$ 的概率保证至少有 $n/2$ 个键具有唯一的哈希值。当 $k$ 趋近于 $n$ 时，该结论的证明具有非平凡性。我们相信，证明此结论所采用的技术可能具有独立的研究价值。我们将新的 IBLT 应用于 Fleischhacker、Larsen 和 Simkin（Eurocrypt 2023）近期研究的加密压缩问题。我们扩展了他们的方法，使其适用于更广泛的加密方案类别，并利用我们新的 IBLT 实现了渐进更优的压缩率。