Query optimization in big data analytics remains computationally expensive, particularly for resource-constrained environments where traditional optimizers fail to satisfy memory and latency constraints. We present an agentic query planning system that combines a rule-based teacher planner, UCB1 bandit exploration, cost-aware prediction, and knowledge distillation to a lightweight student planner. Our teacher planner generates SQL plans using six key optimization strategies, while UCB1 bandit search efficiently explores the plan space under explicit resource constraints. A Random Forest cost model predicts query latency from plan features, enabling cost-aware decisions. A distilled student planner (Logistic Regression or Gradient Boosting) learns to mimic teacher-bandit decisions for fast inference. Evaluation on NYC Taxi and IMDB datasets demonstrates 23% latency reduction compared to default planners while maintaining 94% constraint satisfaction. The student planner achieves 89% accuracy in replicating optimal plans with 15x faster inference time. Our single-file implementation enables reproducible big-data analytics on resource-limited machines and is publicly available at https://github.com/mahdinaser/agentic-kd-planner.
翻译:在大数据分析中,查询优化在计算上仍然成本高昂,尤其是在资源受限环境中,传统优化器无法满足内存和延迟约束。我们提出了一种代理式查询规划系统,该系统结合了基于规则的教师规划器、UCB1赌博机探索、成本感知预测以及向轻量级学生规划器的知识蒸馏。我们的教师规划器利用六种关键优化策略生成SQL计划,而UCB1赌博机搜索则在明确的资源约束下高效探索计划空间。一个随机森林成本模型根据计划特征预测查询延迟,从而实现成本感知决策。经蒸馏的学生规划器(逻辑回归或梯度提升)学习模仿教师-赌博机决策以实现快速推理。在纽约出租车和IMDB数据集上的评估表明,与默认规划器相比,延迟降低了23%,同时保持了94%的约束满足率。学生规划器在复制最优计划时实现了89%的准确率,推理速度提升了15倍。我们的单文件实现方案能够在资源受限的机器上进行可复现的大数据分析,并已在https://github.com/mahdinaser/agentic-kd-planner公开提供。