A package query returns a package - a multiset of tuples - that maximizes or minimizes a linear objective function subject to linear constraints, thereby enabling in-database decision support. Prior work has established the equivalence of package queries to Integer Linear Programs (ILPs) and developed the SketchRefine algorithm for package query processing. While this algorithm was an important first step toward supporting prescriptive analytics scalably inside a relational database, it struggles when the data size grows beyond a few hundred million tuples or when the constraints become very tight. In this paper, we present Progressive Shading, a novel algorithm for processing package queries that can scale efficiently to billions of tuples and gracefully handle tight constraints. Progressive Shading solves a sequence of optimization problems over a hierarchy of relations, each resulting from an ever-finer partitioning of the original tuples into homogeneous groups until the original relation is obtained. This strategy avoids the premature discarding of high-quality tuples that can occur with SketchRefine. Our novel partitioning scheme, Dynamic Low Variance, can handle very large relations with multiple attributes and can dynamically adapt to both concentrated and spread-out sets of attribute values, provably outperforming traditional partitioning schemes such as KD-tree. We further optimize our system by replacing our off-the-shelf optimization software with customized ILP and LP solvers, called Dual Reducer and Parallel Dual Simplex respectively, that are highly accurate and orders of magnitude faster.

翻译：包查询返回一个包（即元组多重集），通过在线性约束条件下最大化或最小化线性目标函数，实现数据库内的决策支持。先前研究已建立包查询与整数线性规划（ILP）的等价性，并开发了用于包查询处理的SketchRefine算法。尽管该算法是支持在关系数据库内可扩展执行规范性分析的重要初步尝试，但当数据规模超过数亿个元组或约束条件变得极为严格时，其性能会显著下降。本文提出Progressive Shading算法，一种新型包查询处理方法，可高效扩展至数十亿元组并优雅应对严格约束。Progressive Shading通过求解一系列基于关系层次结构的优化问题实现，其中每个层次由原始元组依据渐进细化的同质分组策略划分而成，直至还原原始关系。该策略避免了SketchRefine中可能出现的优质元组过早丢弃问题。我们创新的分区方案"动态低方差"（Dynamic Low Variance）能够处理含多属性的大型关系，并动态适应属性值的集中与分散分布，理论上优于KD树等传统分区方案。我们进一步通过定制化ILP与LP求解器（分别称为Dual Reducer和Parallel Dual Simplex）替代现成优化软件来优化系统，这些求解器兼具高精度与数量级的速度提升。