Large-scale cloud systems play a pivotal role in modern IT infrastructure. However, incidents occurring within these systems can lead to service disruptions and adversely affect user experience. To swiftly resolve such incidents, on-call engineers depend on crafting domain-specific language (DSL) queries to analyze telemetry data. However, writing these queries can be challenging and time-consuming. This paper presents a thorough empirical study on the utilization of queries of KQL, a DSL employed for incident management in a large-scale cloud management system at Microsoft. The findings obtained underscore the importance and viability of KQL queries recommendation to enhance incident management. Building upon these valuable insights, we introduce Xpert, an end-to-end machine learning framework that automates KQL recommendation process. By leveraging historical incident data and large language models, Xpert generates customized KQL queries tailored to new incidents. Furthermore, Xpert incorporates a novel performance metric called Xcore, enabling a thorough evaluation of query quality from three comprehensive perspectives. We conduct extensive evaluations of Xpert, demonstrating its effectiveness in offline settings. Notably, we deploy Xpert in the real production environment of a large-scale incident management system in Microsoft, validating its efficiency in supporting incident management. To the best of our knowledge, this paper represents the first empirical study of its kind, and Xpert stands as a pioneering DSL query recommendation framework designed for incident management.

翻译：大型云系统在现代IT基础设施中扮演着关键角色。然而，这些系统中发生的事件可能导致服务中断并对用户体验产生负面影响。为快速解决此类事件，值班工程师需编写领域特定语言（DSL）查询以分析遥测数据。然而，编写这些查询既具挑战性又耗时。本文对微软大规模云管理系统中用于事件管理的DSL——KQL查询的使用情况进行了全面实证研究。研究结果强调了KQL查询推荐在增强事件管理中的重要性及可行性。基于这些宝贵洞见，我们提出Xpert——一个端到端的机器学习框架，用于自动化KQL推荐流程。通过利用历史事件数据和大语言模型，Xpert能够为新增事件生成定制化KQL查询。此外，Xpert引入了一种名为Xcore的新型性能指标，可从三个综合维度全面评估查询质量。我们通过广泛评估证明了Xpert在离线环境中的有效性。值得注意的是，我们将Xpert部署于微软大规模事件管理系统的真实生产环境中，验证了其在支持事件管理方面的效率。据我们所知，本文是首项此类实证研究，而Xpert则是首个专为事件管理设计的开创性DSL查询推荐框架。