The growing popularity of generative language models has amplified interest in interactive methods to guide model outputs. Prompt refinement is considered one of the most effective means to influence output among these methods. We identify several challenges associated with prompting large language models, categorized into data- and model-specific, linguistic, and socio-linguistic challenges. A comprehensive examination of model outputs, including runner-up candidates and their corresponding probabilities, is needed to address these issues. The beam search tree, the prevalent algorithm to sample model outputs, can inherently supply this information. Consequently, we introduce an interactive visual method for investigating the beam search tree, facilitating analysis of the decisions made by the model during generation. We quantitatively show the value of exposing the beam search tree and present five detailed analysis scenarios addressing the identified challenges. Our methodology validates existing results and offers additional insights.

翻译：生成式语言模型的日益普及激发了人们对交互式方法引导模型输出的兴趣。在这些方法中，提示优化被认为是影响输出的最有效手段之一。我们识别出与大语言模型提示相关的若干挑战，并将其分为数据与模型特定、语言学及社会语言学三类。解决这些问题需要对模型输出（包括候选序列及其对应概率）进行深入分析。束搜索树作为模型输出采样的主流算法，天然能够提供这些信息。因此，我们提出一种基于束搜索树的交互式可视化方法，以辅助分析模型生成过程中的决策。我们通过定量实验证明了暴露束搜索树的价值，并针对所识别的挑战提出了五个详细分析场景。该方法既验证了现有结论，又提供了新的洞见。