In a modern DBMS, working memory is frequently the limiting factor when processing in-memory analytic query operations such as joins, sorting, and aggregation. Existing resource estimation approaches for a DBMS estimate the resource consumption of a query by computing an estimate of each individual database operator in the query execution plan. Such an approach is slow and error-prone as it relies upon simplifying assumptions, such as uniformity and independence of the underlying data. Additionally, the existing approach focuses on individual queries separately and does not factor in other queries in the workload that may be executed concurrently. In this research, we are interested in query performance optimization under concurrent execution of a batch of queries (a workload). Specifically, we focus on predicting the memory demand for a workload rather than providing separate estimates for each query within it. We introduce the problem of workload memory prediction and formalize it as a distribution regression problem. We propose Learned Workload Memory Prediction (LearnedWMP) to improve and simplify estimating the working memory demands of workloads. Through a comprehensive experimental evaluation, we show that LearnedWMP reduces the memory estimation error of the state-of-the-practice method by up to 47.6%. Compared to an alternative single-query model, during training and inferencing, the LearnedWMP model and its variants were 3x to 10x faster. Moreover, LearnedWMP-based models were at least 50% smaller in most cases. Overall, the results demonstrate the advantages of the LearnedWMP approach and its potential for a broader impact on query performance optimization.

翻译：在现代数据库管理系统中，处理内存分析查询操作（如连接、排序和聚合）时，工作内存常常成为瓶颈。现有的数据库资源估计方法通过计算查询执行计划中每个数据库算子的估计值来预测查询的资源消耗。这种方法速度慢且易出错，因为它依赖于简化假设（如底层数据的均匀性和独立性）。此外，现有方法仅针对单个查询分别估计，未考虑工作负载中可能并发执行的其他查询。本研究关注批处理查询（工作负载）并发执行下的查询性能优化，重点预测工作负载的内存需求，而非为其中每个查询提供独立估计。我们提出工作负载内存预测问题，并将其形式化为分布回归问题。我们提出学习型工作负载内存预测方法LearnedWMP，以改进并简化工作负载工作内存需求的估计。通过全面的实验评估，我们证明LearnedWMP将现有实践方法的内存估计误差最多降低47.6%。与替代的单查询模型相比，在训练和推理过程中，LearnedWMP模型及其变体速度提升3至10倍。此外，在大多数情况下，基于LearnedWMP的模型体积至少缩小50%。总体而言，实验结果展示了LearnedWMP方法的优势及其对查询性能优化产生更广泛影响的潜力。