Modern large language models (LLMs) represent a paradigm shift in what can plausibly be expected of machine learning models. The fact that LLMs can effectively generate sensible answers to a diverse range of queries suggests that they would be useful in customer support applications. While powerful, LLMs have been observed to be prone to hallucination which unfortunately makes their near term use in customer support applications challenging. To address this issue we present a system that allows us to use an LLM to augment our customer support advocates by re-framing the language modeling task as a discriminative classification task. In this framing, we seek to present the top-K best template responses for a customer support advocate to use when responding to a customer. We present the result of both offline and online experiments where we observed offline gains and statistically significant online lifts for our experimental system. Along the way, we present observed scaling curves for validation loss and top-K accuracy, resulted from model parameter ablation studies. We close by discussing the space of trade-offs with respect to model size, latency, and accuracy as well as and suggesting future applications to explore.

翻译：现代大语言模型（LLMs）代表了机器学习模型能力预期的范式转变。LLMs能够针对多样化查询生成合理回答的事实表明，它们在客户支持应用中具有实用价值。尽管功能强大，但LLMs已被观察到容易产生幻觉，这使其在近期客户支持应用中的使用面临挑战。为解决此问题，我们提出一种系统，通过将语言建模任务重构为判别式分类任务，使LLM能够增强客户支持专员的能力。在此框架下，我们的目标是为客户支持专员在与客户互动时提供最佳的K个模板化回复选项。我们展示了离线和在线实验的结果：实验系统在离线评估中取得性能提升，并在在线测试中获得统计显著的改进效果。通过模型参数消融研究，我们进一步呈现了验证损失与Top-K准确率的缩放曲线。最后，我们探讨了模型规模、延迟与准确率之间的权衡空间，并对未来可探索的应用方向提出建议。