Transformer-based language models (TLMs) have widely been recognized to be a cutting-edge technology for the successful development of deep-learning-based solutions to problems and applications that require natural language processing and understanding. Like for other textual domains, TLMs have indeed pushed the state-of-the-art of AI approaches for many tasks of interest in the legal domain. Despite the first Transformer model being proposed about six years ago, there has been a rapid progress of this technology at an unprecedented rate, whereby BERT and related models represent a major reference, also in the legal domain. This article provides the first systematic overview of TLM-based methods for AI-driven problems and tasks in the legal sphere. A major goal is to highlight research advances in this field so as to understand, on the one hand, how the Transformers have contributed to the success of AI in supporting legal processes, and on the other hand, what are the current limitations and opportunities for further research development.

翻译：基于Transformer的语言模型（TLMs）已被广泛认为是为需要自然语言处理与理解的问题和应用开发深度学习解决方案的前沿技术。与其他文本领域类似，TLMs确实推动了法律领域诸多重要任务的人工智能方法达到最新发展水平。尽管首个Transformer模型约在六年前被提出，但该技术以史无前例的速度取得了快速进展，其中BERT及相关模型已成为重要参考基准，在法律领域亦复如此。本文首次系统性概述了基于TLM的方法在法律领域人工智能驱动问题与任务中的应用。主要目标在于强调该领域的研究进展，以理解两方面内容：一方面，Transformer如何促进了人工智能在法律流程支持中的成功应用；另一方面，当前存在哪些局限性及进一步研究发展的机遇。