A recent line of work in NLP focuses on the (dis)ability of models to generalise compositionally for artificial languages. However, when considering natural language tasks, the data involved is not strictly, or locally, compositional. Quantifying the compositionality of data is a challenging task, which has been investigated primarily for short utterances. We use recursive neural models (Tree-LSTMs) with bottlenecks that limit the transfer of information between nodes. We illustrate that comparing data's representations in models with and without the bottleneck can be used to produce a compositionality metric. The procedure is applied to the evaluation of arithmetic expressions using synthetic data, and sentiment classification using natural language data. We demonstrate that compression through a bottleneck impacts non-compositional examples disproportionately and then use the bottleneck compositionality metric (BCM) to distinguish compositional from non-compositional samples, yielding a compositionality ranking over a dataset.

翻译：国家语言方案最近的工作重点是模型的(残疾)性,以概括人造语言的构成。然而,在考虑自然语言任务时,所涉数据并非严格或局部的构成性。数据构成性量化是一项具有挑战性的任务,主要针对短话进行了调查。我们使用循环神经模型(Tree-LSTMs),其瓶颈限制了节点之间的信息传输。我们说明,比较模型中的数据在有瓶颈和没有瓶颈的模型中的表述可以用来得出一个构成性指标。该程序适用于利用合成数据对算术表达进行评估,以及利用自然语言数据对情绪分类进行评估。我们证明,通过瓶点影响非组合示例进行压缩不相称,然后使用瓶颈构成性指标(BCM)来区分组成与非组合样本,从而得出数据集的构成性等级。