Deep networks typically learn concepts via classifiers, which involves setting up a model and training it via gradient descent to fit the concept-labeled data. We will argue instead that learning a concept could be done by looking at its moment statistics matrix to generate a concrete representation or signature of that concept. These signatures can be used to discover structure across the set of concepts and could recursively produce higher-level concepts by learning this structure from those signatures. When the concepts are `intersected', signatures of the concepts can be used to find a common theme across a number of related `intersected' concepts. This process could be used to keep a dictionary of concepts so that inputs could correctly identify and be routed to the set of concepts involved in the (latent) generation of the input.
翻译:深度网络通常通过分类器学习概念,这需要建立模型并通过梯度下降训练以拟合概念标注数据。相反,我们主张可通过考察概念的矩统计矩阵来生成该概念的具体表示或签名,从而完成概念学习。这些签名可用于发现概念集合中的结构,并可通过从这些签名中学习该结构来递归地产生更高层次的概念。当概念被"交叉"时,可利用概念的签名寻找多个相关"交叉"概念的共同主题。该过程可用于维护概念字典,使输入能够正确识别并路由到参与(潜层)生成该输入的一组概念。