Traditional recurrent neural networks (RNNs) have a fixed, finite number of memory cells. In theory (assuming bounded range and precision), this limits their formal language recognition power to regular languages, and in practice, RNNs have been shown to be unable to learn many context-free languages (CFLs). In order to expand the class of languages RNNs recognize, prior work has augmented RNNs with a nondeterministic stack data structure, putting them on par with pushdown automata and increasing their language recognition power to CFLs. Nondeterminism is needed for recognizing all CFLs (not just deterministic CFLs), but in this paper, we show that nondeterminism and the neural controller interact to produce two more unexpected abilities. First, the nondeterministic stack RNN can recognize not only CFLs, but also many non-context-free languages. Second, it can recognize languages with much larger alphabet sizes than one might expect given the size of its stack alphabet. Finally, to increase the information capacity in the stack and allow it to solve more complicated tasks with large alphabet sizes, we propose a new version of the nondeterministic stack that simulates stacks of vectors rather than discrete symbols. We demonstrate perplexity improvements with this new model on the Penn Treebank language modeling benchmark.

翻译：传统的经常性神经网络(RNN)具有固定的、有限的记忆细胞数量。理论上(假设有约束的射程和精确度),这限制了他们正式的语言识别能力,实际上,RNN无法学习多种无背景语言(CFLs ) 。为了扩大语言类别,先前的工作增加了RNN(RNNs),使其具有非确定性的堆叠数据结构,使其与推压自动数据相对应,并提高其对CFLs的语言识别能力。在理论上(假设有约束的射程和精确度),这限制了他们正式的语言识别能力。在理论上(假设有约束性的CFLs),这限制了他们正式的语言识别能力,但实际上,RNNNN(Cs)无法学习多种无背景语言(CFLs ) 。 首先,非确定性堆叠不仅可以识别CFLS,而且许多非确定性的堆叠数据结构。 其次,它可以识别比人们预期的堆叠字母规模大得多的语言模式。最后,是为了增加堆叠的信息能力,并允许它用大型的、不确定性 CFLFLO控制器操作的更复杂的任务,我们提议用新版本的硬式的硬式的纸式的硬体的硬体模型展示。</s>