Mixed-precision computing has become increasingly important in modern high-performance computing and machine learning applications. When implementing custom mixed-precision functions -- such as fused operators, optimized GPU kernels, or quantized inference paths -- it is critical to verify their numerical accuracy. Traditional approaches typically compare the custom implementation against a reference using a single error metric. However, this single-delta approach provides limited insight into whether the observed errors are inherent to the precision level or specific to the implementation. This paper introduces \textit{Dual-Delta Testing}, a systematic methodology that evaluates two error distributions against a high-precision oracle, enabling rigorous comparison between a custom implementation and a baseline reference. We present the mathematical framework, algorithmic formulation, statistical analysis techniques, and practical examples demonstrating the methodology's effectiveness in evaluating numerical accuracy.
翻译:混合精度计算在现代高性能计算与机器学习应用中日益重要。在实现自定义混合精度函数(如融合算子、优化GPU内核或量化推理路径)时,验证其数值精度至关重要。传统方法通常使用单一误差度量将自定义实现与参考实现进行比较。然而,这种单一差分方法难以区分观测误差是源于精度限制还是实现缺陷。本文提出\textit{双重差分测试},这是一种系统化方法,通过同时评估两个误差分布相对于高精度基准的偏离,实现对自定义实现与基线参考的严格比较。我们介绍了该方法的数学框架、算法形式化、统计分析技术,并通过实际案例验证了该方法在评估数值精度方面的有效性。