Large language models (LLMs) provide a new way to build chatbots by accepting natural language prompts. Yet, it is unclear how to design prompts to power chatbots to carry on naturalistic conversations while pursuing a given goal, such as collecting self-report data from users. We explore what design factors of prompts can help steer chatbots to talk naturally and collect data reliably. To this aim, we formulated four prompt designs with different structures and personas. Through an online study (N = 48) where participants conversed with chatbots driven by different designs of prompts, we assessed how prompt designs and conversation topics affected the conversation flows and users' perceptions of chatbots. Our chatbots covered 79% of the desired information slots during conversations, and the designs of prompts and topics significantly influenced the conversation flows and the data collection performance. We discuss the opportunities and challenges of building chatbots with LLMs.

翻译：大型语言模型（LLMs）为通过接受自然语言提示构建聊天机器人提供了新途径。然而，如何设计提示以使聊天机器人在追求特定目标（如收集用户自报数据）的同时保持自然对话，目前尚不明确。我们探究了哪些提示设计因素有助于引导聊天机器人自然交谈并可靠地收集数据。为此，我们制定了四种具有不同结构和人格的提示设计方案。通过一项在线研究（N=48），参与者与由不同提示设计驱动的聊天机器人进行对话，我们评估了提示设计和对话主题如何影响对话流程及用户对聊天机器人的感知。我们的聊天机器人在对话中覆盖了79%的目标信息槽位，且提示设计和主题对对话流程及数据收集性能有显著影响。我们讨论了利用LLMs构建聊天机器人的机遇与挑战。