Meta-learning usually refers to a learning algorithm that learns from other learning algorithms. The problem of uncertainty in the predictions of neural networks shows that the world is only partially predictable and a learned neural network cannot generalize to its ever-changing surrounding environments. Therefore, the question is how a predictive model can represent multiple predictions simultaneously. We aim to provide a fundamental understanding of learning to learn in the contents of Decentralized Neural Networks (Decentralized NNs) and we believe this is one of the most important questions and prerequisites to building an autonomous intelligence machine. To this end, we shall demonstrate several pieces of evidence for tackling the problems above with Meta Learning in Decentralized NNs. In particular, we will present three different approaches to building such a decentralized learning system: (1) learning from many replica neural networks, (2) building the hierarchy of neural networks for different functions, and (3) leveraging different modality experts to learn cross-modal representations.

翻译：元学习通常指一种从其他学习算法中学习的学习算法。神经网络预测结果的不确定性问题表明，世界只能被部分预测，而学习出的神经网络无法泛化到其不断变化的外部环境中。因此，问题在于预测模型如何同时表示多种预测结果。我们旨在深入理解去中心化神经网络（Decentralized NNs）背景下的学习如何学习，并认为这是构建自主智能机器最为关键的问题和先决条件之一。为此，我们将展示几项证据，表明通过去中心化神经网络中的元学习可以解决上述问题。具体而言，我们将介绍三种构建此类去中心化学习系统的不同方法：（1）从多个副本神经网络中学习，（2）构建不同功能的神经网络层次结构，以及（3）利用不同模态的专家来学习跨模态表示。