User simulation is increasingly vital to develop and evaluate recommender systems (RSs). While Large Language Models (LLMs) offer promising avenues to simulate user behavior, they often struggle with the absence of specific task alignment required for RSs and the efficiency demands of large-scale simulation. A vast yet underutilized resource for enhancing this alignment is the extensive user feedback inherent in RSs, but leveraging it is challenging due to its ambiguity, noise and massive volume, which hinders efficient preference alignment. To overcome these hurdles, we introduce a novel data construction framework that leverages user feedback in RSs with advanced LLM capabilities to generate high-quality simulation data. Our framework unfolds in two key phases: (1) using LLMs to generate decision-making processes as explanatory rationales on simulation samples, thereby reducing ambiguity; and (2) data distillation based on uncertainty estimation and behavior sampling to efficiently filter the most informative, denoised samples. Accordingly, we fine-tune lightweight LLMs, as user simulators, using such high-quality dataset with corresponding decision-making processes. Extensive experiments confirm that our framework significantly boosts the alignment with human preferences and the in-domain reasoning capabilities of the fine-tuned LLMs, providing more insightful and interpretable signals for RS interaction. We believe our work, together with publicly available developed framework, high-quality mixed-domain dataset, and fine-tuned LLM checkpoints, will advance the RS community and offer valuable insights for broader human-centric AI research. Our code is available at https://github.com/Joinn99/UserMirrorer.

翻译：用户模拟对于开发和评估推荐系统愈发关键。尽管大语言模型为模拟用户行为提供了有前景的途径，但它们常面临缺乏推荐系统所需特定任务对齐以及大规模模拟效率需求的难题。推荐系统中固有的广泛用户反馈是增强这种对齐的宝贵但未被充分利用的资源，但由于其模糊性、噪声和大体量，利用它进行高效的偏好对齐颇具挑战。为克服这些障碍，我们提出了一种新颖的数据构建框架，该框架利用推荐系统中的用户反馈与先进的大语言模型能力来生成高质量的模拟数据。该框架包含两个关键阶段：（1）使用大语言模型在模拟样本上生成决策过程作为解释性理由，从而降低模糊性；（2）基于不确定性估计和行为采样的数据蒸馏，以高效筛选最具信息量的去噪样本。据此，我们使用此类高质量数据集及相应决策过程对轻量级大语言模型进行微调，作为用户模拟器。大量实验证实，我们的框架显著提升了微调大语言模型与人类偏好的对齐程度及领域内推理能力，为推荐系统交互提供了更具洞察力和可解释性的信号。我们相信，这项工作连同公开发布的框架、高质量跨领域数据集及微调后的大语言模型检查点，将推动推荐系统社区发展，并为更广泛的人本人工智能研究提供宝贵见解。我们的代码已开源至https://github.com/Joinn99/UserMirrorer。