As Large Language Models (LLMs) evolve from general-purpose assistants to user-centric agents, personalization has become central to aligning model behavior with individual preferences, making the evaluation of personalized alignment a critical bottleneck. Existing evaluation methods-ranging from automatic metrics to LLM-as-a-judge approaches-fail to capture subjective, user-specific preferences embedded in long-term interaction histories. We identify three essential principles for reliable and effective personalized evaluation: Representativeness, User-Consistency, and Discriminativeness. To address these principles, we introduce Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment. Under this paradigm, we propose PARL (Preference-Aware Rubric Learning for Personalized Evaluation), a framework that learns to induce preference-aware evaluation rubrics directly from raw user histories and performs a self-validation mechanism to ensure consistency with the user's preferences. PARL integrates rubric induction with a discriminative reinforcement learning objective that contrasts user-authored responses against competitive personalized model outputs, enabling the learned rubrics to capture precise, user-specific decision boundaries. Experiments on real-world personalized text generation tasks show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users and tasks, while capturing stable stylistic preferences and fine-grained evaluative patterns. To ensure reproducibility, our code is available at https://github.com/SnowCharmQ/PARL.
翻译:随着大型语言模型从通用助手演进为用户中心型智能体,个性化已成为将模型行为与个体偏好对齐的核心环节,而对个性化对齐效果的评估成为关键瓶颈。现有评估方法——从自动化指标到LLM作为裁判方法——均难以捕捉嵌入长期交互历史中的主观性用户特定偏好。我们识别出可靠有效的个性化评估需遵循三项基本原则:代表性、用户一致性和区分性。针对这些原则,我们提出"个性化评估即学习"范式,将个性化评估形式化为学习问题而非静态判断。在此范式下,我们提出PARL(面向个性化评估的偏好感知评分准则学习)框架,该框架通过从原始用户历史中直接学习诱导出偏好感知的评估准则,并采用自验证机制确保与用户偏好的一致性。PARL将准则归纳与判别式强化学习目标相结合,通过对比用户撰写回答与竞争性个性化模型输出,使所学准则能够捕获精确的用户特定决策边界。在真实场景的个性化文本生成任务实验表明,PARL能稳定诱导出高保真度的评估准则,可靠识别与用户对齐的回答,同时跨用户和任务泛化,捕捉稳定的风格偏好与细粒度评估模式。为确保可复现性,我们的代码已开源至 https://github.com/SnowCharmQ/PARL。