In this paper, we propose using deep neural networks to extract important information from Vietnamese legal questions, a fundamental task towards building a question answering system in the legal domain. Given a legal question in natural language, the goal is to extract all the segments that contain the needed information to answer the question. We introduce a deep model that solves the task in three stages. First, our model leverages recent advanced autoencoding language models to produce contextual word embeddings, which are then combined with character-level and POS-tag information to form word representations. Next, bidirectional long short-term memory networks are employed to capture the relations among words and generate sentence-level representations. At the third stage, borrowing ideas from graph-based dependency parsing methods which provide a global view on the input sentence, we use biaffine classifiers to estimate the probability of each pair of start-end words to be an important segment. Experimental results on a public Vietnamese legal dataset show that our model outperforms the previous work by a large margin, achieving 94.79% in the F1 score. The results also prove the effectiveness of using contextual features extracted from pre-trained language models combined with other types of features such as character-level and POS-tag features when training on a limited dataset.

翻译：本文提出使用深度神经网络从越南法律问题中提取重要信息，这是构建法律领域问答系统的基础任务。给定自然语言形式的法律问题，目标是提取所有包含回答问题所需信息的片段。我们引入了一个分三个阶段解决该任务的深度模型。首先，模型利用先进的自动编码语言模型生成上下文词嵌入，然后结合字符级和词性标注信息形成词表示。其次，采用双向长短期记忆网络捕捉词间关系并生成句子级表示。第三阶段，借鉴基于图的依存句法分析方法（能对输入句子提供全局视角），使用双仿射分类器估计每对起始-结束词成为重要片段的概率。在公开的越南法律数据集上的实验结果表明，我们的模型以较大优势超越了先前工作，F1分数达到94.79%。结果还证明，在有限数据集上训练时，从预训练语言模型中提取的上下文特征与字符级、词性标注特征等其他类型特征结合使用的有效性。