The deployment of autonomous agents in environments involving human interaction has increasingly raised security concerns. Consequently, understanding the circumstances behind an event becomes critical, requiring the development of capabilities to justify their behaviors to non-expert users. Such explanations are essential in enhancing trustworthiness and safety, acting as a preventive measure against failures, errors, and misunderstandings. Additionally, they contribute to improving communication, bridging the gap between the agent and the user, thereby improving the effectiveness of their interactions. This work presents an accountability and explainability architecture implemented for ROS-based mobile robots. The proposed solution consists of two main components. Firstly, a black box-like element to provide accountability, featuring anti-tampering properties achieved through blockchain technology. Secondly, a component in charge of generating natural language explanations by harnessing the capabilities of Large Language Models (LLMs) over the data contained within the previously mentioned black box. The study evaluates the performance of our solution in three different scenarios, each involving autonomous agent navigation functionalities. This evaluation includes a thorough examination of accountability and explainability metrics, demonstrating the effectiveness of our approach in using accountable data from robot actions to obtain coherent, accurate and understandable explanations, even when facing challenges inherent in the use of autonomous agents in real-world scenarios.
翻译:在涉及人机交互的环境中部署自主智能体日益引发安全关切。因此,理解事件背后的情境变得至关重要,这要求开发向非专业用户解释其行为的能力。此类解释对于提升可信度与安全性具有关键作用,可作为预防故障、错误和误解的措施;同时有助于改善沟通,弥合智能体与用户之间的认知差距,从而提升交互效能。本研究提出一种专为基于ROS的移动机器人设计的问责与可解释性架构。该方案包含两大核心组件:其一,具有防篡改特性的黑盒式模块,通过区块链技术实现问责功能;其二,利用大语言模型(LLMs)对前述黑盒中数据的处理能力,生成自然语言解释的组件。我们评估了该解决方案在三种不同场景下的性能,每个场景均涉及自主智能体导航功能。评估涵盖对问责性与可解释性指标的全面分析,结果表明:即使在现实场景中面对自主智能体应用存在的固有挑战时,本方法仍能基于机器人行为的可问责数据生成连贯、准确且易于理解的解释。