This paper presents a new learning algorithm, termed Deep Bi-directional Predictive Coding (DBPC) that allows developing networks to simultaneously perform classification and reconstruction tasks using the same weights. Predictive Coding (PC) has emerged as a prominent theory underlying information processing in the brain. The general concept for learning in PC is that each layer learns to predict the activities of neurons in the previous layer which enables local computation of error and in-parallel learning across layers. In this paper, we extend existing PC approaches by developing a network which supports both feedforward and feedback propagation of information. Each layer in the networks trained using DBPC learn to predict the activities of neurons in the previous and next layer which allows the network to simultaneously perform classification and reconstruction tasks using feedforward and feedback propagation, respectively. DBPC also relies on locally available information for learning, thus enabling in-parallel learning across all layers in the network. The proposed approach has been developed for training both, fully connected networks and convolutional neural networks. The performance of DBPC has been evaluated on both, classification and reconstruction tasks using the MNIST and FashionMNIST datasets. The classification and the reconstruction performance of networks trained using DBPC is similar to other approaches used for comparison but DBPC uses a significantly smaller network. Further, the significant benefit of DBPC is its ability to achieve this performance using locally available information and in-parallel learning mechanisms which results in an efficient training protocol. This results clearly indicate that DBPC is a much more efficient approach for developing networks that can simultaneously perform both classification and reconstruction.

翻译：本文提出一种新型学习算法——深度双向预测编码（DBPC），该算法可使网络在保持相同权重的情况下同时执行分类与重建任务。预测编码（PC）已成为解释大脑信息处理机制的重要理论。PC学习的基本理念是：各层通过预测前一层神经元活动实现误差的局部化计算与跨层并行学习。本文通过构建支持前馈与反馈双向信息传播的网络，对现有PC方法进行了拓展。采用DBPC训练的网络中，各层同步学习预测前一层与后一层的神经元活动，从而分别通过前馈传播与反馈传播实现分类与重建的双重任务。DBPC同样依托局部可用信息完成学习，支持网络全部层级的并行训练。该方法已成功应用于全连接网络与卷积神经网络的训练。基于MNIST与FashionMNIST数据集，本文对DBPC在分类与重建任务上的性能进行了评估。实验表明：采用DBPC训练的网络在分类与重建性能上与对比方法相当，但网络规模显著减小。更重要的是，DBPC的核心优势在于能够仅凭局部可用信息与并行学习机制达成该性能水平，从而形成高效的训练范式。这些结果明确表明：DBPC为开发同时具备分类与重建能力的网络提供了一种更为高效的方法。