Scaling training data and model parameters has long driven progress in large language models (LLMs), but this paradigm is increasingly constrained by the scarcity of high-quality data and diminishing returns from rising computational costs. As a result, recent work is increasing the focus on continual learning from real-world deployment, where user interaction logs provide a rich source of authentic human feedback and procedural knowledge. However, learning from user logs is challenging due to their unstructured and noisy nature. Vanilla LLM systems often struggle to distinguish useful feedback signals from noisy user behavior, and the disparity between user log collection and model optimization (e.g., the off-policy optimization problem) further strengthens the problem. To this end, we propose UNO (User log-driveN Optimization), a unified framework for improving LLM systems (LLMsys) with user logs. UNO first distills logs into semi-structured rules and preference pairs, then employs query-and-feedback-driven clustering to manage data heterogeneity, and finally quantifies the cognitive gap between the model's prior knowledge and the log data. This assessment guides the LLMsys to adaptively filter out noisy feedback and construct different modules for primary and reflective experiences extracted from user logs, thereby improving future responses. Extensive experiments show that UNO achieves state-of-the-art effectiveness and efficiency, significantly outperforming Retrieval Augmented Generation (RAG) and memory-based baselines. We have open-sourced our code at https://github.com/bebr2/UNO .

翻译：长期以来，扩大训练数据和模型参数规模一直是大语言模型（LLM）发展的主要驱动力，但这一范式正日益受到高质量数据稀缺以及计算成本上升带来的收益递减的制约。因此，近期研究越来越关注从实际部署中进行持续学习，其中用户交互日志提供了丰富且真实的人类反馈和过程性知识来源。然而，由于用户日志的非结构化和噪声特性，从中学习颇具挑战。传统的大语言模型系统往往难以从嘈杂的用户行为中区分出有用的反馈信号，而用户日志收集与模型优化之间的差异（例如离策略优化问题）进一步加剧了这一难题。为此，我们提出了UNO（用户日志驱动优化），一个利用用户日志改进大语言模型系统（LLMsys）的统一框架。UNO首先将日志提炼为半结构化规则和偏好对，然后采用查询-反馈驱动的聚类方法来管理数据异质性，最后量化模型先验知识与日志数据之间的认知差距。该评估指导LLMsys自适应地过滤噪声反馈，并针对从用户日志中提取的主要经验和反思性经验构建不同的模块，从而改进未来的响应。大量实验表明，UNO在效果和效率上均达到了最先进的水平，显著优于检索增强生成（RAG）和基于记忆的基线方法。我们已在 https://github.com/bebr2/UNO 开源代码。