Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts.

翻译：现代人工智能智能体功能强大，但往往难以与个体用户独特且动态变化的偏好保持一致。现有方法通常依赖于静态数据集，要么在交互历史数据上训练隐式偏好模型，要么将用户画像编码到外部记忆中。然而，这些方法在面对新用户以及偏好随时间变化时表现不佳。本文提出基于人类反馈的个性化智能体（PAHF）框架，该框架通过显式的用户专属记忆，使智能体能够在在线交互中持续学习以实现个性化。PAHF实现了一个三步循环机制：（1）通过行动前澄清以消除歧义，（2）基于从记忆中检索的偏好信息来执行行动，（3）整合行动后反馈以更新记忆，从而应对偏好漂移。为评估该能力，我们设计了一个四阶段评估协议，并在具身操作与在线购物领域构建了两个基准测试。这些基准测试量化了智能体从零开始学习初始偏好，并随后适应角色转变的能力。我们的理论分析与实验结果表明，将显式记忆与双反馈通道相结合至关重要：PAHF的学习速度显著更快，且持续优于无记忆和单通道基线方法，有效降低了初始个性化误差，并能快速适应偏好变化。