Most deep learning pipelines are built on real-valued operations to deal with real-valued inputs such as images, speech or music signals. However, a lot of applications naturally make use of complex-valued signals or images, such as MRI or remote sensing. Additionally the Fourier transform of signals is complex-valued and has numerous applications. We aim to make deep learning directly applicable to these complex-valued signals without using projections into $\mathbb{R}^2$. Thus we add to the recent developments of complex-valued neural networks by presenting building blocks to transfer the transformer architecture to the complex domain. We present multiple versions of a complex-valued Scaled Dot-Product Attention mechanism as well as a complex-valued layer normalization. We test on a classification and a sequence generation task on the MusicNet dataset and show improved robustness to overfitting while maintaining on-par performance when compared to the real-valued transformer architecture.

翻译：大多数深度学习流程基于实数运算来处理实数输入（如图像、语音或音乐信号）。然而，许多应用天然涉及复数信号或图像（如磁共振成像或遥感）。此外，信号的傅里叶变换是复数值的，且具有广泛应用。我们的目标是使深度学习能够直接应用于这些复数信号，而无需将其投影至$\mathbb{R}^2$。为此，我们基于复数神经网络的最新进展，提出了将Transformer架构迁移至复数域的构建模块。我们设计了多个版本的复数缩放点积注意力机制以及复数层归一化方法。在MusicNet数据集上的分类与序列生成任务测试表明，与实数域Transformer架构相比，所提方法在保持同等性能的同时，具有更强的抗过拟合鲁棒性。