Deep neural networks based on layer-stacking architectures have historically suffered from poor inherent interpretability. Meanwhile, symbolic probabilistic models function with clear interpretability, but how to combine them with neural networks to enhance their performance remains to be explored. In this paper, we try to marry these two systems for text classification via a structured language model. We propose a Symbolic-Neural model that can learn to explicitly predict class labels of text spans from a constituency tree without requiring any access to span-level gold labels. As the structured language model learns to predict constituency trees in a self-supervised manner, only raw texts and sentence-level labels are required as training data, which makes it essentially a general constituent-level self-interpretable classification model. Our experiments demonstrate that our approach could achieve good prediction accuracy in downstream tasks. Meanwhile, the predicted span labels are consistent with human rationales to a certain degree.

翻译：基于层堆叠架构的深度神经网络历来缺乏良好的内在可解释性。与此同时，符号概率模型具有清晰的可解释性，但如何将其与神经网络结合以提升性能仍有待探索。本文尝试通过一种结构化语言模型将这两种系统融合于文本分类任务中。我们提出一种符号-神经模型，该模型能够从成分树中显式预测文本片段的类别标签，而无需任何跨度级别的真实标签。由于结构化语言模型以自监督方式学习预测成分树，仅需原始文本和句子级别标签作为训练数据，这使得它本质上成为一个通用的成分级别自解释分类模型。实验表明，我们的方法在下游任务中能实现良好的预测精度。同时，预测的跨度标签与人类依据在某种程度上具有一致性。