Although artificial neural network (ANN) based speech enhancement (SE) methods demonstrate excellent performance, the high computational complexity and high energy consumption hinder their deployment in practical front-end processing tasks.} Currently, the spiking neural networks (SNNs) have shown potential in reducing power consumption. However, the discrete binary activation and complex spatio-temporal dynamics of SNNs often result in information loss. The current challenge therefore focuses on how to maintain performance and reduce computational complexity. To address this issue, this work propose a Dual-Branch Hybrid Neural (DBHN) Network. 1) In terms of network architecture: A dual-branch network integrating ANN and SNN was designed, where the SNN branch reduces power consumption while the ANN branch addresses information loss; The BandSplit and Time-Frequency (TF) -Mamba modules were developed to simultaneously compress energy consumption and enhance model performance; Spiking Feature Extraction Group (SFEG) and Information Transformation Block (ITB) components were implemented with residual connections to mitigate information loss while further refining feature representations. 2) To facilitate inter-branch information fusion: An Interaction module was designed to promote information exchange at various stages of the dual-branch network; A TF-Cross Attention-Fusion module was designed to perform time-frequency domain fusion of dual-branch information while data-adaptively guiding the SNN branch to retain more critical information. Results show that the proposed model maintains superior performance across three public datasets while achieving an average 7.5 fold reduction in computational complexity compared to baseline models.
翻译:尽管基于人工神经网络(ANN)的语音增强方法表现出色,但高计算复杂度与高能耗阻碍了其在实际前端处理任务中的部署。当前,脉冲神经网络(SNN)在降低功耗方面展现出潜力,但SNN的离散二值激活和复杂的时空动态特性常导致信息损失。因此,当前挑战聚焦于如何在保持性能的同时降低计算复杂度。为解决此问题,本文提出双分支混合神经网络(DBHN)。1)网络架构层面:设计了集成ANN与SNN的双分支网络,其中SNN分支降低功耗,ANN分支弥补信息损失;开发了BandSplit与时频-曼巴(TF-Mamba)模块以同步压缩能耗并增强模型性能;采用带残差连接的脉冲特征提取组(SFEG)与信息转换块(ITB)组件,在缓解信息损失的同时进一步细化特征表示。2)跨分支信息融合层面:设计交互模块以促进双分支网络各阶段的信息交换;设计时频交叉注意力融合(TF-Cross Attention-Fusion)模块,对双分支信息进行时频域融合,并数据自适应地引导SNN分支保留更多关键信息。结果表明,所提模型在三个公开数据集上保持优异性能的同时,相较于基线模型实现了平均7.5倍的计算复杂度降幅。