Debugging distributed systems in-production is inevitable and hard. Myriad interactions between concurrent components in modern, complex and large-scale systems cause non-deterministic bugs that offline testing and verification fail to capture. When bugs surface at runtime, their root causes may be far removed from their symptoms. To identify a root cause, developers often need evidence scattered across multiple components and traces. Unfortunately, existing tools fail to quickly and automatically record useful provenance information at low overheads, leaving developers to manually perform the onerous evidence collection task. Lumos is an online debugging framework that exposes application-level bug provenances--the computational history linking symptoms of an incident to their root causes. Lumos leverages dependency-guided instrumentation powered by static analysis to identify program state related to a bug's provenance, and exposes them via lightweight on-demand recording. Lumos provides developers with enough evidence to identify a bug's root cause, while incurring low runtime overhead, and given only a few occurrences of a bug.

翻译：摘要：生产环境下的分布式系统调试既不可避免又充满挑战。现代复杂大规模系统中并发组件间的海量交互会导致非确定性缺陷，而这些缺陷难以通过离线测试与验证手段捕获。当缺陷在运行时显现时，其根本原因可能远离症状表面。为定位根本原因，开发人员通常需要跨多个组件和追踪记录收集证据。遗憾的是，现有工具无法在低开销条件下快速自动记录有效的溯源信息，迫使开发人员手动完成繁重的证据收集任务。Lumos是一个在线调试框架，能够暴露应用级缺陷溯源信息——即连接事件症状与根本原因的计算追溯路径。该框架通过静态分析驱动的依赖导向插桩技术，识别与缺陷溯源相关的程序状态，并借助轻量级按需记录机制将其暴露出来。Lumos能在仅需少量缺陷出现次数的情况下，为开发人员提供足以定位缺陷根本原因的证据，同时保持较低的运行时开销。