This work addresses meta-learning (ML) by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units results in sparse representations from each model layer, as the units are organized into blocks where only one unit generates a non-zero output. The main operating principle of the introduced units rely on stochastic principles, as the network performs posterior sampling over competing units to select the winner. Therefore, the proposed networks are explicitly designed to extract input data representations of sparse stochastic nature, as opposed to the currently standard deterministic representation paradigm. Our approach produces state-of-the-art predictive accuracy on few-shot image classification and regression experiments, as well as reduced predictive error on an active learning setting; these improvements come with an immensely reduced computational cost.

翻译：这项工作涉及元学习(ML),方法是考虑与当地随机性赢家全取(LWTA)激活的深网络。这类网络单位导致每个模型层的表达形式很少,因为每个模型层的单位组成成区块,其中只有一个单位产生非零产出。引入单元的主要操作原则依赖于随机原则,因为网络对竞争的单位进行后继取样,以选择获胜者。因此,拟议的网络的设计明确是为了提取稀有的随机性输入数据,而不是目前的标准确定性代表模式。我们的方法在微小图像分类和回归实验方面产生最先进的预测准确性,并在积极的学习环境中减少预测错误;这些改进带来了巨大的计算成本。