The black-box nature of deep learning models in NLP hinders their widespread application. The research focus has shifted to Hierarchical Attribution (HA) for its ability to model feature interactions. Recent works model non-contiguous combinations with a time-costly greedy search in Eculidean spaces, neglecting underlying linguistic information in feature representations. In this work, we introduce a novel method, namely Poincar\'e Explanation (PE), for modeling feature interactions using hyperbolic spaces in an $O(n^2logn)$ time complexity. Inspired by Poincar\'e model, we propose a framework to project the embeddings into hyperbolic spaces, which exhibit better inductive biases for syntax and semantic hierarchical structures. Eventually, we prove that the hierarchical clustering process in the projected space could be viewed as building a minimum spanning tree and propose a time efficient algorithm. Experimental results demonstrate the effectiveness of our approach.
翻译:深度学习模型在自然语言处理中的黑箱特性阻碍了其广泛应用。研究焦点已转向层级归因(Hierarchical Attribution, HA),因其能够建模特征交互。近期研究在欧几里得空间中采用耗时的贪心搜索处理非连续组合,却忽略了特征表示中蕴含的语言学信息。本文提出一种名为庞加莱解释(Poincaré Explanation, PE)的新方法,利用双曲空间以$O(n^2logn)$时间复杂度建模特征交互。受庞加莱模型启发,我们提出一个框架将嵌入向量投影至双曲空间,该空间对句法与语义层级结构具有更优的归纳偏置。最终,我们证明投影空间中的层级聚类过程可视为构建最小生成树,并据此提出一种高效算法。实验结果表明了本方法的有效性。