While current deep learning algorithms have been successful for a wide variety of artificial intelligence (AI) tasks, including those involving structured image data, they present deep neurophysiological conceptual issues due to their reliance on the gradients that are computed by backpropagation of errors (backprop). Gradients are required to obtain synaptic weight adjustments but require knowledge of feed-forward activities in order to conduct backward propagation, a biologically implausible process. This is known as the "weight transport problem". Therefore, in this work, we present a more biologically plausible approach towards solving the weight transport problem for image data. This approach, which we name the error kernel driven activation alignment (EKDAA) algorithm, accomplishes through the introduction of locally derived error transmission kernels and error maps. Like standard deep learning networks, EKDAA performs the standard forward process via weights and activation functions; however, its backward error computation involves adaptive error kernels that propagate local error signals through the network. The efficacy of EKDAA is demonstrated by performing visual-recognition tasks on the Fashion MNIST, CIFAR-10 and SVHN benchmarks, along with demonstrating its ability to extract visual features from natural color images. Furthermore, in order to demonstrate its non-reliance on gradient computations, results are presented for an EKDAA trained CNN that employs a non-differentiable activation function.
翻译:尽管当前的深度学习算法在包括结构化图像数据处理在内的广泛人工智能任务中取得了成功,但由于其依赖通过误差反向传播(反向传播)计算的梯度,因此存在深刻的神经生理学概念问题。梯度是获取突触权重调整所必需的,但需要知道前馈活动才能进行反向传播,这一过程在生物学上缺乏合理性,即所谓的“权重传输问题”。因此,本文提出了一种更具生物学合理性的方法来解决图像数据的权重传输问题。该方法名为误差核驱动激活对齐(EKDAA)算法,通过引入局部导出的误差传输核和误差图来实现。与标准深度学习网络类似,EKDAA通过权重和激活函数执行标准前向过程;但其反向误差计算涉及自适应误差核,通过网络传播局部误差信号。通过在Fashion MNIST、CIFAR-10和SVHN基准数据集上执行视觉识别任务,以及展示其从自然彩色图像中提取视觉特征的能力,证明了EKDAA的有效性。此外,为证明其对梯度计算的非依赖性,本文展示了采用不可微激活函数的EKDAA训练卷积神经网络(CNN)所获得的结果。