The necessity for interpretability in natural language processing (NLP) has risen alongside the growing prominence of large language models. Among the myriad tasks within NLP, text generation stands out as a primary objective of autoregressive models. The NLP community has begun to take a keen interest in gaining a deeper understanding of text generation, leading to the development of model-agnostic explainable artificial intelligence (xAI) methods tailored to this task. The design and evaluation of explainability methods are non-trivial since they depend on many factors involved in the text generation process, e.g., the autoregressive model and its stochastic nature. This paper outlines 17 challenges categorized into three groups that arise during the development and assessment of attribution-based explainability methods. These challenges encompass issues concerning tokenization, defining explanation similarity, determining token importance and prediction change metrics, the level of human intervention required, and the creation of suitable test datasets. The paper illustrates how these challenges can be intertwined, showcasing new opportunities for the community. These include developing probabilistic word-level explainability methods and engaging humans in the explainability pipeline, from the data design to the final evaluation, to draw robust conclusions on xAI methods.

翻译：自然语言处理（NLP）中可解释性的必要性随着大语言模型的日益突出而提升。在NLP的众多任务中，文本生成作为自回归模型的主要目标尤为突出。NLP社区已开始对深入理解文本生成产生浓厚兴趣，从而推动了针对该任务的模型无关可解释人工智能（xAI）方法的发展。可解释性方法的设计与评估并非易事，因其依赖于文本生成过程中涉及的诸多因素，例如自回归模型及其随机性本质。本文概述了在基于归因的可解释性方法开发与评估过程中出现的三大类共17项挑战。这些挑战涵盖分词问题、解释相似性定义、词元重要性与预测变化度量、所需人工干预程度，以及合适测试数据集的创建。论文展示了这些挑战如何相互关联，并揭示了社区面临的新机遇，包括开发概率性词级可解释性方法，以及将人类参与融入从数据设计到最终评估的可解释性流程，从而对xAI方法得出稳健结论。