LLMs have advanced tool-using agents for real-world applications, yet they often lead to unexpected behaviors or results. Beyond obvious failures, the subtle issue of "intent deviation" severely hinders reliable evaluation and performance improvement. Existing post-training methods generally leverage either real system samples or virtual data simulated by LLMs. However, the former is costly due to reliance on hand-crafted user requests, while the latter suffers from distribution shift from the real tools in the wild. Additionally, both methods lack negative samples tailored to intent deviation scenarios, hindering effective guidance on preference learning. We introduce RISE, a "Real-to-Virtual" method designed to mitigate intent deviation. Anchoring on verified tool primitives, RISE synthesizes virtual trajectories and generates diverse negative samples through mutation on critical parameters. With synthetic data, RISE fine-tunes backbone LLMs via the two-stage training for intent alignment. Evaluation results demonstrate that data synthesized by RISE achieve promising results in eight metrics covering user requires, execution trajectories and agent responses. Integrating with training, RISE achieves an average 35.28% improvement in Acctask (task completion) and 23.27% in Accintent (intent alignment), outperforming SOTA baselines by 1.20--42.09% and 1.17--54.93% respectively.
翻译:大型语言模型(LLM)推动了面向实际应用的工具使用智能体的发展,但它们常常导致意外行为或结果。除明显故障外,“意图偏离”这一微妙问题严重阻碍了可靠的评估与性能提升。现有的后训练方法通常利用真实系统样本或由LLM模拟的虚拟数据。然而,前者因依赖人工编写的用户请求而成本高昂,后者则因与真实工具分布存在偏移而受限。此外,两种方法均缺乏针对意图偏离场景定制的负样本,从而阻碍了偏好学习的有效引导。我们提出了RISE,一种旨在缓解意图偏离的“真实到虚拟”方法。RISE以已验证的工具原语为锚点,合成虚拟轨迹,并通过关键参数变异生成多样化的负样本。利用合成数据,RISE通过两阶段训练对骨干LLM进行微调以实现意图对齐。评估结果表明,RISE合成的数据在涵盖用户需求、执行轨迹和智能体响应的八项指标上均取得了良好结果。结合训练过程,RISE在Acctask(任务完成度)上平均提升35.28%,在Accintent(意图对齐度)上平均提升23.27%,分别优于SOTA基线1.20–42.09%和1.17–54.93%。