Existing dialogue models may encounter scenarios which are not well-represented in the training data, and as a result generate responses that are unnatural, inappropriate, or unhelpful. We propose the "Ask an Expert" framework in which the model is trained with access to an "expert" which it can consult at each turn. Advice is solicited via a structured dialogue with the expert, and the model is optimized to selectively utilize (or ignore) it given the context and dialogue history. In this work the expert takes the form of an LLM. We evaluate this framework in a mental health support domain, where the structure of the expert conversation is outlined by pre-specified prompts which reflect a reasoning strategy taught to practitioners in the field. Blenderbot models utilizing "Ask an Expert" show quality improvements across all expert sizes, including those with fewer parameters than the dialogue model itself. Our best model provides a $\sim 10\%$ improvement over baselines, approaching human-level scores on "engingingness" and "helpfulness" metrics.

翻译：现有对话模型在训练数据中可能遇到未充分体现的场景，从而生成不自然、不恰当或无帮助的响应。我们提出"向专家请教"框架，通过训练模型使其能够访问一个"专家"，并在每个对话回合进行咨询。通过与该专家的结构化对话获取建议，模型被优化为根据上下文和对话历史选择性利用（或忽略）这些建议。在本研究中，专家采用大型语言模型（LLM）的形式。我们在心理健康支持领域评估该框架，其中专家对话的结构由预定义提示词规定，这些提示词反映了该领域从业者被教导的推理策略。使用"向专家请教"的Blenderbot模型在所有专家规模下均展现出质量提升，包括参数少于对话模型本身的设置。我们的最佳模型相比基线提升约$\sim 10\%$，在"参与度"和"帮助性"指标上接近人类水平分数。