Context: Recent developments in natural language processing have facilitated the adoption of chatbots in typically collaborative software engineering tasks (such as diagram modelling). Families of experiments can assess the performance of tools and processes and, at the same time, alleviate some of the typical shortcomings of individual experiments (e.g., inaccurate and potentially biased results due to a small number of participants). Objective: Compare the usability of a chatbot for collaborative modelling (i.e., SOCIO) and an online web tool (i.e., Creately). Method: We conducted a family of three experiments to evaluate the usability of SOCIO against the Creately online collaborative tool in academic settings. Results: The student participants were faster at building class diagrams using the chatbot than with the online collaborative tool and more satisfied with SOCIO. Besides, the class diagrams built using the chatbot tended to be more concise -albeit slightly less complete. Conclusion: Chatbots appear to be helpful for building class diagrams. In fact, our study has helped us to shed light on the future direction for experimentation in this field and lays the groundwork for researching the applicability of chatbots in diagramming.

翻译：背景：自然语言处理的最新进展促进了聊天机器人在典型协作式软件工程任务（如图表建模）中的应用。系列实验能够评估工具与流程的性能，同时缓解单个实验的某些典型缺陷（例如，因参与者数量较少导致结果不准确且可能存在偏差）。目标：比较协作建模聊天机器人（即SOCIO）与在线网页工具（即Creately）的可用性。方法：我们在学术环境中进行了三项实验组成的系列研究，以评估SOCIO相对于Creately在线协作工具的可用性。结果：学生参与者使用聊天机器人构建类图的速度快于在线协作工具，且对SOCIO的满意度更高。此外，使用聊天机器人构建的类图往往更简洁——尽管完整性略低。结论：聊天机器人似乎有助于构建类图。事实上，我们的研究有助于揭示该领域实验的未来方向，并为研究聊天机器人在图表绘制中的适用性奠定基础。