The use of Transformer represents a recent success in speech enhancement. However, as its core component, self-attention suffers from quadratic complexity, which is computationally prohibited for long speech recordings. Moreover, it allows each time frame to attend to all time frames, neglecting the strong local correlations of speech signals. This study presents a simple yet effective sparse self-attention for speech enhancement, called ripple attention, which simultaneously performs fine- and coarse-grained modeling for local and global dependencies, respectively. Specifically, we employ local band attention to enable each frame to attend to its closest neighbor frames in a window at fine granularity, while employing dilated attention outside the window to model the global dependencies at a coarse granularity. We evaluate the efficacy of our ripple attention for speech enhancement on two commonly used training objectives. Extensive experimental results consistently confirm the superior performance of the ripple attention design over standard full self-attention, blockwise attention, and dual-path attention (Sep-Former) in terms of speech quality and intelligibility.

翻译：Transformer的引入代表了语音增强领域的最新成功。然而，其核心组件自注意力机制存在二次复杂度问题，在长语音录音中计算代价过高。此外，该机制允许每个时间帧关注所有其他时间帧，忽视了语音信号中强烈的局部相关性。本研究提出一种简单而有效的稀疏自注意力方法——涟漪注意力，该方法可分别对局部和全局依赖关系进行细粒度与粗粒度建模。具体而言，我们采用局部频带注意力使每个帧以细粒度方式关注其窗口内最近邻的帧，同时在窗口外使用扩张注意力以粗粒度方式建模全局依赖关系。我们在两种常用训练目标上评估了所提涟漪注意力在语音增强中的有效性。大量实验结果表明，与标准全自注意力、分块注意力及双路径注意力（Sep-Former）相比，涟漪注意力设计在语音质量和可懂度方面均具有优越性能。