Transformers have recently achieved state-of-the-art performance in speech separation. These models, however, are computationally demanding and require a lot of learnable parameters. This paper explores Transformer-based speech separation with a reduced computational cost. Our main contribution is the development of the Resource-Efficient Separation Transformer (RE-SepFormer), a self-attention-based architecture that reduces the computational burden in two ways. First, it uses non-overlapping blocks in the latent space. Second, it operates on compact latent summaries calculated from each chunk. The RE-SepFormer reaches a competitive performance on the popular WSJ0-2Mix and WHAM! datasets in both causal and non-causal settings. Remarkably, it scales significantly better than the previous Transformer-based architectures in terms of memory and inference time, making it more suitable for processing long mixtures.

翻译：变压器（Transformer）最近在语音分离领域取得了最先进的性能。然而，这些模型计算量大且需要大量可学习参数。本文探索了基于Transformer的低计算成本语音分离方法。我们的主要贡献是开发了资源高效分离变压器（RE-SepFormer），这是一种基于自注意力机制的架构，通过两种方式减少计算负担。首先，它在潜在空间中使用非重叠块。其次，它基于每个块计算出的紧凑潜在摘要进行运算。RE-SepFormer在流行的WSJ0-2Mix和WHAM！数据集上的因果和非因果设置中均达到了有竞争力的性能。值得注意的是，它在内存和推理时间方面显著优于之前的基于Transformer的架构，更适合处理长混合信号。