Meta-learning usually refers to a learning algorithm that learns from other learning algorithms. The problem of uncertainty in the predictions of neural networks shows that the world is only partially predictable and a learned neural network cannot generalize to its ever-changing surrounding environments. Therefore, the question is how a predictive model can represent multiple predictions simultaneously. We aim to provide a fundamental understanding of learning to learn in the contents of Decentralized Neural Networks (Decentralized NNs) and we believe this is one of the most important questions and prerequisites to building an autonomous intelligence machine. To this end, we shall demonstrate several pieces of evidence for tackling the problems above with Meta Learning in Decentralized NNs. In particular, we will present three different approaches to building such a decentralized learning system: (1) learning from many replica neural networks, (2) building the hierarchy of neural networks for different functions, and (3) leveraging different modality experts to learn cross-modal representations.

翻译：元学习通常指一种从其他学习算法中学习的学习算法。神经网络预测中的不确定性问题表明，世界只是部分可预测的，而学习到的神经网络无法泛化到其不断变化的环境。因此，问题在于预测模型如何能同时表示多种预测。我们旨在深入理解去中心化神经网络（Decentralized NNs）中“学习如何学习”的本质，并认为这是构建自主智能机器最重要的问题和前提之一。为此，我们将展示用去中心化神经网络中的元学习解决上述问题的若干证据。具体而言，我们将介绍三种构建此类去中心化学习系统的不同方法：（1）从多个副本神经网络中学习，（2）构建针对不同功能的神经网络层级结构，以及（3）利用不同模态专家来学习跨模态表征。