Existing work on large language model (LLM) personalization assigned different responding roles to LLM, but overlooked the diversity of questioners. In this work, we propose a new form of questioner-aware LLM personalization, generating different responses even for the same query from different questioners. We design a dual-tower model architecture with a cross-questioner general encoder and a questioner-specific encoder. We further apply contrastive learning with multi-view augmentation, pulling close the dialogue representations of the same questioner, while pulling apart those of different questioners. To mitigate the impact of question diversity on questioner-contrastive learning, we cluster the dialogues based on question similarity and restrict the scope of contrastive learning within each cluster. We also build a multi-questioner dataset from English and Chinese scripts and WeChat records, called MQDialog, containing 173 questioners and 12 responders. Extensive evaluation with different metrics shows a significant improvement in the quality of personalized response generation.
翻译:现有的大语言模型个性化研究主要关注为模型分配不同的响应角色,但忽略了提问者的多样性。本研究提出一种新的提问者感知的大语言模型个性化范式,即使对于来自不同提问者的相同查询,也能生成差异化的响应。我们设计了一种双塔模型架构,包含跨提问者通用编码器和提问者特定编码器。进一步采用多视图增强的对比学习方法,拉近同一提问者的对话表征,同时推远不同提问者的对话表征。为降低问题多样性对提问者对比学习的影响,我们基于问题相似度对对话进行聚类,并将对比学习范围限制在每个聚类内部。此外,我们从英文/中文剧本和微信记录中构建了包含173位提问者和12位应答者的多提问者数据集MQDialog。采用多种指标的实验评估表明,该方法在个性化响应生成质量上取得了显著提升。