Distributed systems must frequently keep track of many different types of performance metrics across many different computers. For example, the latency distribution of certain operations may be computed for a large combination of computers, users, and operations. These empirical distributions need to be collected at minimal expense on the individual software components, efficiently aggregated across multiple dimensions, and stored in a compact representation for a variety of downstream data analysis applications. We describe an information loss metric for binned data that allows us to optimize cost of information loss from different histogram representations. We explore the use of polynomial histograms where each bin of a histogram is annotated with moments of the underlying distribution in that bin. These polynomial histograms are compared to traditional histograms using the same storage cost for additional bins instead of annotations in each bin. We describe an application of these techniques for file system metrics for a large production system, and analytically characterize when polynomial histograms offer more information at lower cost.

翻译：分布式系统必须频繁追踪众多计算机上的多种性能指标。例如，特定操作的延迟分布可能需要针对计算机、用户和操作的大量组合进行计算。这些经验分布需要在各个软件组件上以最小代价收集，跨多个维度高效聚合，并以紧凑表示形式存储，以支持各种下游数据分析应用。我们描述了一种针对分箱数据的 信息损失度量，该度量允许我们优化不同直方图表示带来的信息损失成本。我们探索了多项式直方图的使用，其中直方图的每个箱都用该箱内基础分布的矩进行标注。将这些多项式直方图与传统直方图进行比较，后者使用相同的存储成本来增加额外箱而非在箱内添加标注。我们描述了这些技术在一个大型生产系统的文件系统度量中的应用，并通过分析定性了多项式直方图以更低成本提供更多信息的场景。