Most applications of machine learning to classification assume a closed set of balanced classes. This is at odds with the real world, where class occurrence statistics often follow a long-tailed power-law distribution and it is unlikely that all classes are seen in a single sample. Nonparametric Bayesian models naturally capture this phenomenon, but have significant practical barriers to widespread adoption, namely implementation complexity and computational inefficiency. To address this, we present a method for extracting the inductive bias from a nonparametric Bayesian model and transferring it to an artificial neural network. By simulating data with a nonparametric Bayesian prior, we can metalearn a sequence model that performs inference over an unlimited set of classes. After training, this "neural circuit" has distilled the corresponding inductive bias and can successfully perform sequential inference over an open set of classes. Our experimental results show that the metalearned neural circuit achieves comparable or better performance than particle filter-based methods for inference in these models while being faster and simpler to use than methods that explicitly incorporate Bayesian nonparametric inference.

翻译：机器学习在分类中的大多数应用都假设一个封闭的平衡类别集合。这与现实世界不符，因为现实中类别的出现统计量通常遵循长尾幂律分布，且所有类别不太可能在单个样本中全部出现。非参数贝叶斯模型自然能够捕捉这一现象，但在广泛应用中面临重大实际障碍，即实现复杂性和计算低效。为解决此问题，我们提出了一种从非参数贝叶斯模型中提取归纳偏置并将其迁移至人工神经网络的方法。通过使用非参数贝叶斯先验模拟数据，我们可以元学习一个序列模型，该模型能对无限类别集合进行推断。训练后，这个“神经回路”已提炼出相应的归纳偏置，并能成功对开放类别集合进行序列推断。实验结果表明，与基于粒子滤波的方法相比，元学习的神经回路在这些模型的推断中实现了相当或更优的性能，同时比显式结合贝叶斯非参数推断的方法更快、更易使用。