Sequence-to-Sequence (S2S) models have achieved remarkable success on various text generation tasks. However, learning complex structures with S2S models remains challenging as external neural modules and additional lexicons are often supplemented to predict non-textual outputs. We present a systematic study of S2S modeling using contained decoding on four core tasks: part-of-speech tagging, named entity recognition, constituency and dependency parsing, to develop efficient exploitation methods costing zero extra parameters. In particular, 3 lexically diverse linearization schemas and corresponding constrained decoding methods are designed and evaluated. Experiments show that although more lexicalized schemas yield longer output sequences that require heavier training, their sequences being closer to natural language makes them easier to learn. Moreover, S2S models using our constrained decoding outperform other S2S approaches using external resources. Our best models perform better than or comparably to the state-of-the-art for all 4 tasks, lighting a promise for S2S models to generate non-sequential structures.

翻译：序列到序列（S2S）模型已在多种文本生成任务中取得显著成功。然而，利用S2S模型学习复杂结构仍具挑战性，因为预测非文本输出时通常需要额外补充外部神经模块和词典资源。本文针对四项核心任务——词性标注、命名实体识别、成分句法分析和依存句法分析——系统研究了基于受限解码的S2S建模方法，旨在开发零额外参数的高效利用策略。具体而言，我们设计并评估了三种词汇化程度不同的线性化方案及其对应的受限解码方法。实验表明：虽然高词汇化方案导致输出序列更长且训练负担更重，但其接近自然语言的特性使其更易学习。此外，采用我们受限解码的S2S模型在性能上优于使用外部资源的其他S2S方法。在全部四项任务中，我们的最优模型均达到或优于当前最优水平，这为S2S模型生成非序列化结构开辟了前景。