The recent developments in Large Language Models (LLM), mark a significant moment in the research and development of social interactions with artificial agents. These agents are widely deployed in a variety of settings, with potential impact on users. However, the study of social interactions with agents powered by LLM is still emerging, limited by access to the technology and to data, the absence of standardised interfaces, and challenges to establishing controlled experimental setups using the currently available business-oriented platforms. To answer these gaps, we developed LEXI, LLMs Experimentation Interface, an open-source tool enabling the deployment of artificial agents powered by LLM in social interaction behavioural experiments. Using a graphical interface, LEXI allows researchers to build agents, and deploy them in experimental setups along with forms and questionnaires while collecting interaction logs and self-reported data. The outcomes of usability testing indicate LEXI's broad utility, high usability and minimum mental workload requirement, with distinctive benefits observed across disciplines. A proof-of-concept study exploring the tool's efficacy in evaluating social HAIs was conducted, resulting in high-quality data. A comparison of empathetic versus neutral agents indicated that people perceive empathetic agents as more social, and write longer and more positive messages towards them.

翻译：近年来，大语言模型（LLM）的发展标志着与人工智能体进行社会交互的研究与开发进入了一个重要阶段。这些智能体被广泛应用于各种场景，对用户具有潜在影响。然而，与基于LLM的智能体进行社会交互的研究仍处于起步阶段，其发展受到技术及数据获取途径有限、缺乏标准化接口，以及在使用当前商业导向平台时难以建立受控实验环境等挑战的制约。为弥补这些不足，我们开发了LEXI（LLMs Experimentation Interface），这是一个开源工具，能够在社会交互行为实验中部署基于LLM的人工智能体。通过图形界面，LEXI允许研究人员构建智能体，并将其与表单及问卷一同部署在实验环境中，同时收集交互日志和自我报告数据。可用性测试结果表明，LEXI具有广泛的适用性、高可用性及较低的心智负荷需求，并在不同学科领域展现出独特优势。一项验证该工具在评估社会性人机交互中有效性的概念验证研究已成功开展，并获得了高质量数据。对共情型与中性智能体的比较表明，人们认为共情型智能体更具社会性，并向其发送更长、更积极的信息。