Large language models (LLMs) are widely deployed in various downstream tasks, e.g., auto-completion, aided writing, or chat-based text generation. However, the considered output candidates of the underlying search algorithm are under-explored and under-explained. We tackle this shortcoming by proposing a tree-in-the-loop approach, where a visual representation of the beam search tree is the central component for analyzing, explaining, and adapting the generated outputs. To support these tasks, we present generAItor, a visual analytics technique, augmenting the central beam search tree with various task-specific widgets, providing targeted visualizations and interaction possibilities. Our approach allows interactions on multiple levels and offers an iterative pipeline that encompasses generating, exploring, and comparing output candidates, as well as fine-tuning the model based on adapted data. Our case study shows that our tool generates new insights in gender bias analysis beyond state-of-the-art template-based methods. Additionally, we demonstrate the applicability of our approach in a qualitative user study. Finally, we quantitatively evaluate the adaptability of the model to few samples, as occurring in text-generation use cases.

翻译：大型语言模型（LLM）已广泛部署于各类下游任务，如自动补全、辅助写作或基于聊天的文本生成。然而，其底层搜索算法所考虑的候选输出并未得到充分探索与解释。针对这一不足，我们提出了一种树环内方法，将波束搜索树的视觉表示作为分析、解释并调整生成输出的核心组件。为支持上述任务，我们提出可视化分析技术generAItor，通过为中央波束搜索树附加多种任务专用组件，提供定向可视化与交互操作。该方法支持多层次交互，构建涵盖生成、探索、比较候选输出及基于调整数据微调模型的迭代式流程。案例研究表明，本工具能在性别偏见分析中产生超越现有模板方法的全新见解。此外，我们通过定性用户研究验证了方法的适用性。最后，针对文本生成场景中常见的少量样本情况，对模型的自适应能力进行了定量评估。