We propose a novel, brain-inspired deep neural network model known as the Deep Oscillatory Neural Network (DONN). Deep neural networks like the Recurrent Neural Networks indeed possess sequence processing capabilities but the internal states of the network are not designed to exhibit brain-like oscillatory activity. With this motivation, the DONN is designed to have oscillatory internal dynamics. Neurons of the DONN are either nonlinear neural oscillators or traditional neurons with sigmoidal or ReLU activation. The neural oscillator used in the model is the Hopf oscillator, with the dynamics described in the complex domain. Input can be presented to the neural oscillator in three possible modes. The sigmoid and ReLU neurons also use complex-valued extensions. All the weight stages are also complex-valued. Training follows the general principle of weight change by minimizing the output error and therefore has an overall resemblance to complex backpropagation. A generalization of DONN to convolutional networks known as the Oscillatory Convolutional Neural Network is also proposed. The two proposed oscillatory networks are applied to a variety of benchmark problems in signal and image/video processing. The performance of the proposed models is either comparable or superior to published results on the same data sets.

翻译：我们提出了一种新颖的、受大脑启发的深度神经网络模型，称为深度振荡神经网络（DONN）。像循环神经网络这样的深度神经网络确实具备序列处理能力，但其网络内部状态并非设计为展现类脑的振荡活动。基于此动机，DONN被设计为具有振荡的内部动力学。DONN的神经元要么是非线性神经振荡器，要么是采用Sigmoid或ReLU激活的传统神经元。模型中使用的神经振荡器是Hopf振荡器，其动力学在复数域中描述。输入可以以三种可能模式呈现给神经振荡器。Sigmoid和ReLU神经元也使用复数扩展。所有权重阶段同样是复数值的。训练遵循通过最小化输出误差来改变权重的一般原则，因此总体上与复数反向传播算法相似。我们还提出了DONN向卷积网络的一种推广，称为振荡卷积神经网络。所提出的两种振荡网络被应用于信号和图像/视频处理中的多种基准问题。所提出模型的性能与相同数据集上已发表的结果相比，要么相当，要么更优。