Beam Tree Recursive Neural Network (BT-RvNN) was recently proposed as a simple extension of Gumbel Tree RvNN and it was shown to achieve state-of-the-art length generalization performance in ListOps while maintaining comparable performance on other tasks. However, although not the worst in its kind, BT-RvNN can be still exorbitantly expensive in memory usage. In this paper, we identify the main bottleneck in BT-RvNN's memory usage to be the entanglement of the scorer function and the recursive cell function. We propose strategies to remove this bottleneck and further simplify its memory usage. Overall, our strategies not only reduce the memory usage of BT-RvNN by $10$-$16$ times but also create a new state-of-the-art in ListOps while maintaining similar performance in other tasks. In addition, we also propose a strategy to utilize the induced latent-tree node representations produced by BT-RvNN to turn BT-RvNN from a sentence encoder of the form $f:\mathbb{R}^{n \times d} \rightarrow \mathbb{R}^{d}$ into a sequence contextualizer of the form $f:\mathbb{R}^{n \times d} \rightarrow \mathbb{R}^{n \times d}$. Thus, our proposals not only open up a path for further scalability of RvNNs but also standardize a way to use BT-RvNNs as another building block in the deep learning toolkit that can be easily stacked or interfaced with other popular models such as Transformers and Structured State Space models.

翻译：束树递归神经网络（BT-RvNN）最近被提出作为Gumbel树RvNN的简单扩展，并在ListOps任务中展现了最先进的长度泛化性能，同时在其他任务上保持相当的表现。然而，尽管并非同类中最差，BT-RvNN在内存使用上仍然极其昂贵。本文指出BT-RvNN内存使用的主要瓶颈在于评分函数与递归单元函数之间的耦合。我们提出策略以消除这一瓶颈并进一步简化其内存使用。总体而言，我们的策略不仅将BT-RvNN的内存使用降低了10至16倍，还在ListOps任务上创造了新的最先进水平，同时在其他任务上保持相似性能。此外，我们还提出一种策略，利用BT-RvNN诱导的潜在树节点表示，将BT-RvNN从形式为$f:\mathbb{R}^{n \times d} \rightarrow \mathbb{R}^{d}$的句子编码器转换为形式为$f:\mathbb{R}^{n \times d} \rightarrow \mathbb{R}^{n \times d}$的序列上下文化器。因此，我们的方案不仅为RvNN的进一步可扩展性开辟了道路，还标准化了将BT-RvNN作为深度学习工具包中另一构建模块的使用方式，使其易于与其他流行模型（如Transformer和结构化状态空间模型）堆叠或接口对接。