Root Cause Analysis (RCA) is becoming increasingly crucial for ensuring the reliability of microservice systems. However, performing RCA on modern microservice systems can be challenging due to their large scale, as they usually comprise hundreds of components, leading significant human effort. This paper proposes TraceDiag, an end-to-end RCA framework that addresses the challenges for large-scale microservice systems. It leverages reinforcement learning to learn a pruning policy for the service dependency graph to automatically eliminates redundant components, thereby significantly improving the RCA efficiency. The learned pruning policy is interpretable and fully adaptive to new RCA instances. With the pruned graph, a causal-based method can be executed with high accuracy and efficiency. The proposed TraceDiag framework is evaluated on real data traces collected from the Microsoft Exchange system, and demonstrates superior performance compared to state-of-the-art RCA approaches. Notably, TraceDiag has been integrated as a critical component in the Microsoft M365 Exchange, resulting in a significant improvement in the system's reliability and a considerable reduction in the human effort required for RCA.
翻译:根因分析(RCA)对于确保微服务系统的可靠性变得日益关键。然而,在当代微服务系统上进行RCA因其大规模特性而充满挑战——这些系统通常包含数百个组件,需要大量人工投入。本文提出TraceDiag,一个专为大规模微服务系统设计的端到端RCA框架。该框架利用强化学习为服务依赖图学习剪枝策略,自动消除冗余组件,从而显著提升RCA效率。所学得的剪枝策略具有可解释性,并能完全自适应于新的RCA实例。基于剪枝后的依赖图,可执行高精度、高效率的因果分析方法。该TraceDiag框架在微软Exchange系统的真实数据轨迹上进行了评估,与当前最先进的RCA方法相比展现出卓越性能。值得注意的是,TraceDiag已作为关键组件集成至微软M365 Exchange系统中,显著提升了系统可靠性并大幅减少了RCA所需的人工投入。