Legal case retrieval, which aims to find relevant cases for a query case, plays a core role in the intelligent legal system. Despite the success that pre-training has achieved in ad-hoc retrieval tasks, effective pre-training strategies for legal case retrieval remain to be explored. Compared with general documents, legal case documents are typically long text sequences with intrinsic logical structures. However, most existing language models have difficulty understanding the long-distance dependencies between different structures. Moreover, in contrast to the general retrieval, the relevance in the legal domain is sensitive to key legal elements. Even subtle differences in key legal elements can significantly affect the judgement of relevance. However, existing pre-trained language models designed for general purposes have not been equipped to handle legal elements. To address these issues, in this paper, we propose SAILER, a new Structure-Aware pre-traIned language model for LEgal case Retrieval. It is highlighted in the following three aspects: (1) SAILER fully utilizes the structural information contained in legal case documents and pays more attention to key legal elements, similar to how legal experts browse legal case documents. (2) SAILER employs an asymmetric encoder-decoder architecture to integrate several different pre-training objectives. In this way, rich semantic information across tasks is encoded into dense vectors. (3) SAILER has powerful discriminative ability, even without any legal annotation data. It can distinguish legal cases with different charges accurately. Extensive experiments over publicly available legal benchmarks demonstrate that our approach can significantly outperform previous state-of-the-art methods in legal case retrieval.

翻译：法律案例检索旨在为待查案例寻找相关案例，在智能法律系统中扮演核心角色。尽管预训练在即席检索任务中取得了成功，但针对法律案例检索的有效预训练策略仍有待探索。与通用文档相比，法律案例文档通常为包含内在逻辑结构的长文本序列。然而，现有语言模型难以理解不同结构间的长距离依赖关系。此外，与通用检索不同，法律领域中的相关性对关键法律要素高度敏感，这些要素的细微差异可显著影响相关性判断。然而，现有通用预训练语言模型尚未具备处理法律要素的能力。为解决这些问题，本文提出SAILER——一种面向法律案例检索的新型结构感知预训练语言模型。其创新体现在以下三方面：（1）SAILER充分利用法律案例文档中的结构信息，并像法律专家浏览法律案例文档一样，更加关注关键法律要素。（2）SAILER采用非对称编码器-解码器架构来整合多种不同的预训练目标，从而将跨任务的丰富语义信息编码为密集向量。（3）即使没有任何法律标注数据，SAILER仍具备强大的判别能力，能准确区分不同案由的法律案例。在公开法律基准上的大量实验表明，本方法在法律案例检索中显著优于现有最先进方法。