Estimating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of $n$ items from a data universe equipped with a total order, the task is to compute a sketch (data structure) of size polylogarithmic in $n$. Given the sketch and a query item $y$, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than or equal to $y$. Most works to date focused on additive $\varepsilon n$ error approximation, culminating in the KLL sketch that achieved optimal asymptotic behavior. This paper investigates multiplicative $(1\pm\varepsilon)$-error approximations to the rank. Practical motivation for multiplicative error stems from demands to understand the tails of distributions, and hence for sketches to be more accurate near extreme values. The most space-efficient algorithms due to prior work store either $O(\log(\varepsilon^2 n)/\varepsilon^2)$ or $O(\log^3(\varepsilon n)/\varepsilon)$ universe items. We present a randomized sketch storing $O(\log^{1.5}(\varepsilon n)/\varepsilon)$ items that can $(1\pm\varepsilon)$-approximate the rank of each universe item with high constant probability; this space bound is within an $O(\sqrt{\log(\varepsilon n)})$ factor of optimal. Our algorithm does not require prior knowledge of the stream length and is fully mergeable, rendering it suitable for parallel and distributed computing environments.
翻译:估计流式数据中的秩、分位数和分布是数据分析和监控中的核心任务。给定一个从具有全序关系的数据域中获取的包含 $n$ 个数据项的流,目标是构建一个大小为 $\mathrm{poly}\log(n)$ 的草图(数据结构)。利用该草图和查询项 $y$,应能近似其在流中的秩,即流中小于或等于 $y$ 的元素数量。迄今为止,大多数工作聚焦于加性 $\varepsilon n$ 误差近似,最终以KLL草图实现了最优渐近行为。本文研究秩的乘性 $(1\pm\varepsilon)$-误差近似。乘性误差的实际动机源于对分布尾部理解的需求,因此要求草图在极值附近具有更高精度。先前工作中空间效率最高的算法存储了 $O(\log(\varepsilon^2 n)/\varepsilon^2)$ 或 $O(\log^3(\varepsilon n)/\varepsilon)$ 个数据域元素。我们提出一种随机草图,仅存储 $O(\log^{1.5}(\varepsilon n)/\varepsilon)$ 个元素,即可高常数概率地实现每个数据域元素秩的 $(1\pm\varepsilon)$-近似;该空间复杂度与最优值仅差 $O(\sqrt{\log(\varepsilon n)})$ 因子。我们的算法无需预先知道流长度,且完全可合并,适用于并行和分布式计算环境。