Static and dynamic computational graphs represent two distinct approaches to constructing deep learning frameworks. The former prioritizes compiler-based optimizations, while the latter focuses on programmability and user-friendliness. The recent release of PyTorch 2.0, which supports compiling arbitrary deep learning programs in Python, signifies a new direction in the evolution of deep learning infrastructure to incorporate compiler techniques in a more dynamic manner and support more dynamic language features like dynamic control flows and closures. Given PyTorch's seamless integration with Python, its compiler aims to support arbitrary deep learning code written in Python. However, the inherent dynamism of Python poses challenges to the completeness and robustness of the compiler. While recent research has introduced fuzzing to test deep learning compilers, there is still a lack of comprehensive analysis on how to test dynamic features. To address this issue, we propose several code transformations to generate test cases involving dynamic features. These transformations preserve the program's semantics, ensuring that any discrepancy between the transformed and original programs indicates the presence of a bug. Through our approach, we have successfully identified twenty previously unknown bugs in the PyTorch compiler and its underlying tensor compiler Triton.

翻译：静态与动态计算图代表了构建深度学习框架的两种不同方法。前者侧重于基于编译器的优化，而后者则强调可编程性与用户友好性。近期发布的PyTorch 2.0支持编译Python中任意深度学习程序，标志着深度学习基础设施演进的新方向——以更动态的方式融入编译器技术，并支持动态控制流与闭包等动态语言特性。鉴于PyTorch与Python的无缝集成，其编译器旨在支持用Python编写的任意深度学习代码。然而，Python固有的动态性对编译器的完备性和鲁棒性提出了挑战。尽管近期研究已将模糊测试引入深度学习编译器测试领域，但关于如何测试动态特性的全面分析仍然缺失。针对此问题，我们提出了多项代码变换方法，用于生成涉及动态特性的测试用例。这些变换保留了程序的语义，确保变换后程序与原程序之间的任何差异均指示了缺陷的存在。通过该方法，我们成功识别了PyTorch编译器及其底层张量编译器Triton中二十个此前未知的缺陷。