In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a two-layer Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on the CoNLL-2000 dataset reveal a notable improvement over existing unsupervised methods, enhancing phrase F1 score by up to 6 percentage points. Further, finetuning with downstream tasks results in an additional performance improvement. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model's downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.

翻译：在自然语言处理中，预测句法结构（如句法分析和组块分析）大多依赖人工标注的句法结构。本文提出一种无监督组块分析方法，该句法任务涉及以非层次方式对词语进行分组。我们设计了一个双层层次化循环神经网络，用于建模从词语到组块、从组块到句子的组合过程。我们的方法包含两阶段训练：先通过无监督句法分析器进行预训练，再在下游NLP任务上进行微调。在CoNLL-2000数据集上的实验表明，该方法相较于现有无监督方法有显著提升，短语F1分数最多提高6个百分点。此外，通过下游任务微调可进一步获得性能提升。有趣的是，我们观察到在神经网络模型的下游任务训练过程中，组块结构的出现具有瞬态性。本研究推动了无监督句法结构发现的发展，并为语言学理论的研究开辟了新方向。