Data analytics increasingly runs on distributed lakehouse systems, where platform operators must optimise monetary, resource, and environmental costs. Query Performance Prediction (QPP) helps to balance these costs and supports workload management techniques, such as adaptive resource scaling and low-carbon scheduling. However, runtimes in lakehouses can vary substantially, and the impact of runtime variance on QPP accuracy and workload orchestration has not previously been systematically studied for lakehouse systems. This paper addresses this gap by investigating the runtime variance observed for distributed lakehouse analytical queries and its impact on QPP. First, we quantify the run-to-run variance using Kubernetes deployments across three public clouds and one private cloud, spanning multiple database scales and three analytical benchmarks. Our results demonstrate that repeated executions of the same query can vary in runtime by nearly twofold. Second, we conduct a factor analysis study assessing key sources of this runtime variance such as data locality, co-tenant load, and caching effects. Third, we examine how variance influences state-of-the-art QPP models, revealing that addressing key sources of variance can reduce prediction error up to 80%. Finally, we demonstrate the downstream implications for low-carbon scheduling as an example of a workload management technique that relies on performance prediction, showing that accounting for runtime variance can lead to a significant reduction in carbon costs.

翻译：数据分析越来越多地在分布式湖仓系统上运行，平台运营商必须优化货币、资源和环境成本。查询性能预测（QPP）有助于平衡这些成本，并支持工作负载管理技术，如自适应资源扩缩和低碳调度。然而，湖仓中的运行时可能存在显著差异，而运行时方差对QPP准确性和工作负载编排的影响此前尚未针对湖仓系统进行系统性研究。本文通过调查分布式湖仓分析查询中观察到的运行时方差及其对QPP的影响来填补这一空白。首先，我们使用Kubernetes部署在三个公有云和一个私有云上，跨多个数据库规模和三个分析基准，量化了运行间方差。结果表明，相同查询的重复执行在运行时上可能存在近两倍的差异。其次，我们进行了因子分析研究，评估了这种运行时方差的关键来源，如数据局部性、共租负载和缓存效应。第三，我们检验了方差如何影响最先进的QPP模型，揭示解决关键方差来源可将预测误差降低高达80%。最后，我们以低碳调度为例，展示了其作为依赖性能预测的工作负载管理技术的下游影响，表明考虑运行时方差可显著降低碳成本。