Estimating quantiles, like the median or percentiles, is a fundamental task in data mining and data science. A (streaming) quantile summary is a data structure that can process a set S of n elements in a streaming fashion and at the end, for any phi in (0,1], return a phi-quantile of S up to an eps error, i.e., return a phi'-quantile with phi'=phi +- eps. We are particularly interested in comparison-based summaries that only compare elements of the universe under a total ordering and are otherwise completely oblivious of the universe. The best known deterministic quantile summary is the 20-year old Greenwald-Khanna (GK) summary that uses O((1/eps) log(eps n)) space [SIGMOD'01]. This bound was recently proved to be optimal for all deterministic comparison-based summaries by Cormode and Vesle\'y [PODS'20]. In this paper, we study weighted quantiles, a generalization of the quantiles problem, where each element arrives with a positive integer weight which denotes the number of copies of that element being inserted. The only known method of handling weighted inputs via GK summaries is the naive approach of breaking each weighted element into multiple unweighted items and feeding them one by one to the summary, which results in a prohibitively large update time (proportional to the maximum weight of input elements). We give the first non-trivial extension of GK summaries for weighted inputs and show that it takes O((1/eps) log(eps n)) space and O(log(1/eps)+ log log(eps n)) update time per element to process a stream of length n (under some quite mild assumptions on the range of weights and eps). En route to this, we also simplify the original GK summaries for unweighted quantiles.
翻译:估计分位数(如中位数或百分位数)是数据挖掘与数据科学中的基本任务。(流式)分位数摘要是一种数据结构,能够以流式方式处理包含n个元素的集合S,并最终针对任意φ∈(0,1]返回S的φ-分位数,误差不超过ε,即返回φ' = φ ± ε的φ'-分位数。我们特别关注基于比较的摘要方法,此类方法仅在全序关系下比较论域中的元素,除此之外对论域完全无感知。目前已知最优的确定性分位数摘要是已有20年历史的格林沃尔德-卡纳(GK)摘要,其空间复杂度为O((1/ε) log(ε n)) [SIGMOD'01]。该界近期被Cormode与Veselý证明是所有确定性基于比较摘要方法的最优解 [PODS'20]。本文研究加权分位数——分位数问题的推广形式,其中每个元素附带一个正整数权重,表示该元素被插入的副本数量。处理加权输入时,GK摘要唯一已知的方法是朴素地将每个加权元素拆分为多个无权重单元,并逐一输入摘要,这会导致过大的更新时间(与输入元素的最大权重成正比)。我们首次给出了GK摘要对加权输入的非平凡扩展,证明其在处理长度为n的流时(在权重与ε取值范围的温和假设下)空间复杂度为O((1/ε) log(ε n)),每个元素的更新时间复杂度为O(log(1/ε) + log log(ε n))。在此过程中,我们还简化了原始GK摘要对无权重分位数的处理方式。