Personalizing large language models (LLMs) for individual users has become increasingly important as they are progressively integrated into real-world applications to support users' daily lives. However, existing personalization approaches often fail to distinguish which components of model predictions and training data truly reflect user preferences, leading to superficial personalization alignment. In this paper, we introduce NextQuill, a novel LLM personalization alignment framework grounded in causal preference modeling. We approach personalization from a causal perspective, treating both model predictions and ground-truth data generation as outcomes influenced by user preferences, along with other factors. We define the true preference effect as the causal impact of user history (which reflects preferences) on each token prediction or data generation instance, estimated through causal intervention techniques. Building on this insight, NextQuill introduces two complementary alignment strategies: (1) aligning model-internal causal preference effects on predictions with those reflected in ground-truth data, rather than indiscriminately fitting predictions, and (2) focusing on fitting preference-bearing tokens identified via ground-truth data preference effects, rather than treating all tokens uniformly. By integrating these strategies, NextQuill shifts the alignment process toward learning from causal preference effects, facilitating more effective and personalized adaptation. Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality, offering a principled, causal foundation for LLM personalization. Our codes are available on https://github.com/juntaoyou/NextQuill.

翻译：针对个体用户进行大语言模型（LLM）的个性化定制，在LLM逐步融入实际应用以支持用户日常生活的进程中愈发重要。然而，现有个性化方法往往难以区分模型预测与训练数据中的哪些成分真正反映用户偏好，导致个性化对齐流于表面。本文提出NextQuill——一种基于因果偏好建模的新型LLM个性化对齐框架。我们从因果视角审视个性化问题，将模型预测与真实数据生成均视为受用户偏好及其他因素共同影响的结果，将真实偏好效应定义为用户历史（反映偏好）对每个词元预测或数据生成实例的因果影响，并通过因果干预技术进行估计。基于此，NextQuill引入两种互补的对齐策略：（1）将模型内部的因果偏好效应对预测的影响与真实数据中反映的偏好效应进行对齐，而非不加区分地拟合预测结果；（2）专注于拟合通过真实数据偏好效应识别出的偏好承载词元，而非对所有词元一视同仁。通过整合这些策略，NextQuill将对齐过程转向从因果偏好效应中学习，从而实现更有效、更个性化的自适应。在多个个性化基准上的实验表明，NextQuill显著提升了个性化质量，为LLM个性化提供了基于因果原理的坚实基础。我们的代码已开源在https://github.com/juntaoyou/NextQuill。