Transformer-based language models (TLMs) have widely been recognized to be a cutting-edge technology for the successful development of deep-learning-based solutions to problems and applications that require natural language processing and understanding. Like for other textual domains, TLMs have indeed pushed the state-of-the-art of AI approaches for many tasks of interest in the legal domain. Despite the first Transformer model being proposed about six years ago, there has been a rapid progress of this technology at an unprecedented rate, whereby BERT and related models represent a major reference, also in the legal domain. This article provides the first systematic overview of TLM-based methods for AI-driven problems and tasks in the legal sphere. A major goal is to highlight research advances in this field so as to understand, on the one hand, how the Transformers have contributed to the success of AI in supporting legal processes, and on the other hand, what are the current limitations and opportunities for further research development.

翻译：基于Transformer的语言模型（TLMs）已被广泛认为是解决需要自然语言处理与理解的深度学习方法及应用的尖端技术。与其他文本领域类似，TLMs确实推动了法律领域诸多重要任务的人工智能方法达到最新技术水平。尽管首个Transformer模型提出至今约六年，该技术以前所未有的速度快速发展，其中BERT及相关模型已成为重要标杆，在法律领域同样如此。本文首次系统综述了TLM方法在人工智能驱动的法律问题与任务中的应用，主要目标是突出该领域的研究进展，以期一方面理解Transformer如何助力人工智能在法律流程支持中取得成功，另一方面揭示当前存在的局限性与未来研究发展的机遇。