Customers reach out to online live chat agents with various intents, such as asking about product details or requesting a return. In this paper, we propose the problem of predicting user intent from browsing history and address it through a two-stage approach. The first stage classifies a user's browsing history into high-level intent categories. Here, we represent each browsing history as a text sequence of page attributes and use the ground-truth class labels to fine-tune pretrained Transformers. The second stage provides a large language model (LLM) with the browsing history and predicted intent class to generate fine-grained intents. For automatic evaluation, we use a separate LLM to judge the similarity between generated and ground-truth intents, which closely aligns with human judgments. Our two-stage approach yields significant performance gains compared to generating intents without the classification stage.
翻译:客户出于多种意图联系在线聊天客服,例如询问产品详情或申请退货。本文提出从浏览历史预测用户意图的问题,并通过两阶段方法解决该问题。第一阶段将用户的浏览历史分类为高层意图类别。在此阶段,我们将每条浏览历史表示为页面属性的文本序列,并利用真实类别标签对预训练的Transformer模型进行微调。第二阶段向大型语言模型(LLM)提供浏览历史及预测的意图类别,以生成细粒度意图。在自动评估方面,我们使用独立的LLM评判生成意图与真实意图的相似度,该评估方法与人工判断高度一致。相较于省略分类阶段直接生成意图的方法,我们的两阶段方案取得了显著的性能提升。