We present a notion of bilinear stability, which is to numerical stability what bilinear complexity is to time complexity. In bilinear complexity, an algorithm for evaluating a bilinear operator $\beta : \mathbb{U} \times \mathbb{V} \to \mathbb{W}$ is a decomposition $\beta = \varphi_1 \otimes \psi_1 \otimes w_1 + \dots + \varphi_r \otimes \psi_r \otimes w_r $; the number of terms $r$ captures the speed of the algorithm; and its smallest possible value, i.e., the tensor rank of $\beta$, quantifies the speed of a fastest algorithm. Bilinear stability introduces norms to the mix: The growth factor of the algorithm $\lVert \varphi_1 \rVert_* \lVert \psi_1 \rVert_* \lVert w_1 \rVert + \dots + \lVert \varphi_r \rVert_* \lVert \psi_r \rVert_* \lVert w_r \rVert$ captures the accuracy of the algorithm; and its smallest possible value, i.e., the tensor nuclear norm of $\beta$, quantifies the accuracy of a stablest algorithm. To substantiate this notion, we establish a bound for the forward error in terms of the growth factor and present numerical evidence comparing various fast algorithms for matrix and complex multiplications, showing that larger growth factors correlate with less accurate results. Compared to similar studies of numerical stability, bilinear stability is more general, applying to any bilinear operators and not just matrix or complex multiplications; is more simplistic, bounding forward error in terms of a single (growth) factor; and is truly tensorial like bilinear complexity, invariant under any orthogonal change of coordinates. As an aside, we study a new algorithm for computing complex multiplication in terms of real, much like Gauss's, but is optimally fast and stable in that it attains both tensor rank and nuclear norm.
翻译:我们提出了一种双线性稳定性的概念,其之于数值稳定性正如双线性复杂度之于时间复杂度。在双线性复杂度中,评估双线性算子 $\beta : \mathbb{U} \times \mathbb{V} \to \mathbb{W}$ 的算法可分解为 $\beta = \varphi_1 \otimes \psi_1 \otimes w_1 + \dots + \varphi_r \otimes \psi_r \otimes w_r$;项数 $r$ 刻画了算法的速度,其最小可能值(即 $\beta$ 的张量秩)量化了最快算法的速度。双线性稳定性将范数引入该框架:算法的增长因子 $\lVert \varphi_1 \rVert_* \lVert \psi_1 \rVert_* \lVert w_1 \rVert + \dots + \lVert \varphi_r \rVert_* \lVert \psi_r \rVert_* \lVert w_r \rVert$ 刻画了算法的精度,其最小可能值(即 $\beta$ 的张量核范数)量化了最稳定算法的精度。为验证该概念,我们建立了基于增长因子的前向误差界,并提供了矩阵乘法和复数乘法的多种快速算法的数值实验证据,表明更大的增长因子与更不精确的结果相关。与同类数值稳定性研究相比,双线性稳定性具有更普适性(适用于任意双线性算子,不限于矩阵或复数乘法)、更简洁性(仅通过单一增长因子界定前向误差),以及如双线性复杂度般的张量特性(在任意正交坐标变换下保持不变)。作为附加结果,我们研究了类似高斯方法的基于实数计算复数乘法的新算法,该算法具有最优速度与稳定性,同时达到了张量秩与核范数。