Supervised and unsupervised learning using deep neural networks typically aims to exploit the underlying structure in the training data; this structure is often explained using a latent generative process that produces the data, and the generative process is often hierarchical, involving latent concepts. Despite the significant work on understanding the learning of the latent structure and underlying concepts using theory and experiments, a framework that mathematically captures the definition of a concept and provides ways to operate on concepts is missing. In this work, we propose to characterize a simple primitive concept by the zero set of a collection of polynomials and use moment statistics of the data to uniquely represent the concepts; we show how this view can be used to obtain a signature of the concept. These signatures can be used to discover a common structure across the set of concepts and could recursively produce the signature of higher-level concepts from the signatures of lower-level concepts. To utilize such desired properties, we propose a method by keeping a dictionary of concepts and show that the proposed method can learn different types of hierarchical structures of the data.

翻译：使用深度神经网络的监督与无监督学习通常旨在利用训练数据中的底层结构；这种结构常通过生成数据的潜在生成过程来解释，而该生成过程往往是层次化的，涉及潜在概念。尽管已有大量研究通过理论和实验来理解潜在结构与底层概念的学习过程，但一个能够数学化捕捉概念定义并提供概念操作方法的框架仍然缺失。在本工作中，我们提出通过多项式集合的零点集来表征简单基元概念，并利用数据的矩统计量来唯一表示概念；我们展示了如何利用这一视角获取概念的特征签名。这些签名可用于发现概念集合间的共同结构，并能递归地从低层概念的签名生成高层概念的签名。为利用这些理想特性，我们提出一种通过维护概念字典的方法，并证明所提方法能够学习数据的不同类型层次结构。