The rise of powerful large language models (LLMs) brings about tremendous opportunities for innovation but also looming risks for individuals and society at large. We have reached a pivotal moment for ensuring that LLMs and LLM-infused applications are developed and deployed responsibly. However, a central pillar of responsible AI -- transparency -- is largely missing from the current discourse around LLMs. It is paramount to pursue new approaches to provide transparency for LLMs, and years of research at the intersection of AI and human-computer interaction (HCI) highlight that we must do so with a human-centered perspective: Transparency is fundamentally about supporting appropriate human understanding, and this understanding is sought by different stakeholders with different goals in different contexts. In this new era of LLMs, we must develop and design approaches to transparency by considering the needs of stakeholders in the emerging LLM ecosystem, the novel types of LLM-infused applications being built, and the new usage patterns and challenges around LLMs, all while building on lessons learned about how people process, interact with, and make use of information. We reflect on the unique challenges that arise in providing transparency for LLMs, along with lessons learned from HCI and responsible AI research that has taken a human-centered perspective on AI transparency. We then lay out four common approaches that the community has taken to achieve transparency -- model reporting, publishing evaluation results, providing explanations, and communicating uncertainty -- and call out open questions around how these approaches may or may not be applied to LLMs. We hope this provides a starting point for discussion and a useful roadmap for future research.
翻译:强大的大型语言模型(LLMs)的崛起为创新带来了巨大机遇,但也给个人和社会整体带来了潜在风险。我们正处于确保LLMs及注入LLM的应用程序得以负责任开发和部署的关键时刻。然而,当前围绕LLMs的讨论中,负责任AI的核心支柱——透明度——在很大程度上缺失。追求为LLMs提供透明度的新方法至关重要,而人工智能与人机交互(HCI)交叉领域多年的研究强调,我们必须以人为中心的角度来做到这一点:透明度从根本上关乎支持适当的人类理解,而这种理解是由不同利益相关者在不同情境中出于不同目标所寻求的。在这个LLM的新时代,我们必须通过考虑新兴LLM生态系统中利益相关者的需求、正在构建的新型注入LLM的应用程序类型,以及围绕LLMs的新使用模式和挑战,来开发和设计透明度方法,同时借鉴人们如何处理、互动和利用信息的经验教训。我们反思了为LLMs提供透明度所面临的独特挑战,以及从HCI和负责任AI研究中汲取的、以人为中心视角看待AI透明度的经验教训。随后,我们阐述了学界通常采用的四种实现透明度的方法——模型报告、发布评估结果、提供解释以及沟通不确定性,并指出了这些方法在LLMs上是否适用及其方式相关的开放性问题。我们希望这能为讨论提供一个起点,并为未来研究提供一份有用的路线图。