Generative Large Language Models (LLMs) show potential in data analysis, yet their full capabilities remain uncharted. Our work explores the capabilities of LLMs for creating and refining visualizations via conversational interfaces. We used an LLM to conduct a re-analysis of a prior Wizard-of-Oz study examining the use of chatbots for conducting visual analysis. We surfaced the strengths and weaknesses of LLM-driven analytic chatbots, finding that they fell short in supporting progressive visualization refinements. From these findings, we developed AI Threads, a multi-threaded analytic chatbot that enables analysts to proactively manage conversational context and improve the efficacy of its outputs. We evaluate its usability through a crowdsourced study (n=40) and in-depth interviews with expert analysts (n=10). We further demonstrate the capabilities of AI Threads on a dataset outside the LLM's training corpus. Our findings show the potential of LLMs while also surfacing challenges and fruitful avenues for future research.

翻译：生成式大型语言模型（LLM）在数据分析中展现出潜力，但其完整能力仍有待探索。本研究探讨了LLM通过对话式界面创建和优化可视化能力。我们利用LLM对先前一项关于聊天机器人辅助可视化分析的Wizard-of-Oz实验进行了再分析，揭示了LLM驱动的分析型聊天机器人的优势与不足，发现其在支持渐进式可视化优化方面存在局限。基于这些发现，我们开发了AI Threads——一种多线程分析型聊天机器人，使分析师能够主动管理对话上下文并提升输出有效性。通过众包研究（n=40）与专家分析师深度访谈（n=10）评估了其可用性，并在LLM训练语料之外的测试集上验证了AI Threads的能力。研究结果既展现了LLM的潜力，也揭示了未来研究的挑战与可行方向。