Deep Learning (DL) libraries, such as PyTorch, are widely used for building and deploying DL models on various hardware platforms. Meanwhile, they are found to contain bugs that lead to incorrect calculation results and cause issues like non-convergence training and inaccurate prediction of DL models. Thus, many efforts have been made to test DL libraries and reveal bugs. However, existing DL library testing methods manifest limitations: model-level testing methods cause complexity in fault localization. Meanwhile, API-level testing methods often generate invalid inputs or primarily focus on extreme inputs that lead to crash failures; they also ignore testing realistic API interactions. These limitations may lead to missing detection of bugs, even in the frequently used APIs. To address these limitations, we propose SORT (Subgraph-Oriented Realistic Testing) to differential test DL libraries on different hardware platforms. SORT takes popular API interaction patterns, represented as frequent subgraphs of model computation graphs, as test subjects. In this way, it introduces realistic API interaction sequences while maintaining efficiency in locating faulty APIs for observed errors. Besides, SORT prepares test inputs by referring to extensive features of runtime inputs for each API in executing real-life benchmark data. The generated inputs are expected to better simulate such valid real inputs and reveal bugs more likely to happen in real-life usage. Evaluation on 728 frequent subgraphs of 49 popular PyTorch models demonstrates that SORT achieves a 100\% valid input generation rate, detects more precision bugs than existing methods, and reveals interaction-related bugs missed by single-API testing. 18 precision bugs in PyTorch are identified.

翻译：深度学习（DL）库，如 PyTorch，被广泛用于在各种硬件平台上构建和部署深度学习模型。同时，人们发现这些库中包含可能导致计算结果错误的缺陷，并引发深度学习模型训练不收敛、预测不准确等问题。因此，已有许多工作致力于测试深度学习库并揭示其缺陷。然而，现有的深度学习库测试方法存在局限性：模型级测试方法导致故障定位复杂；而 API 级测试方法常常生成无效输入，或主要关注导致崩溃故障的极端输入；它们也忽略了测试现实的 API 交互。这些局限性可能导致即使在频繁使用的 API 中也会遗漏缺陷检测。为应对这些局限性，我们提出了 SORT（面向子图的现实测试），用于在不同硬件平台上对深度学习库进行差分测试。SORT 将流行的 API 交互模式（表示为模型计算图的频繁子图）作为测试对象。通过这种方式，它在为观察到的错误定位故障 API 时保持高效的同时，引入了现实的 API 交互序列。此外，SORT 通过参考执行真实基准数据时每个 API 运行时输入的广泛特征来准备测试输入。生成的输入有望更好地模拟此类有效的真实输入，并揭示更可能在现实使用中发生的缺陷。对 49 个流行 PyTorch 模型中 728 个频繁子图的评估表明，SORT 实现了 100% 的有效输入生成率，比现有方法检测到更多的精度缺陷，并揭示了单 API 测试遗漏的与交互相关的缺陷。在 PyTorch 中识别出 18 个精度缺陷。