In recent years, a variety of powerful agentic workflows have been applied to solve a wide range of human problems. However, existing workflow orchestration still faces key challenges, including high manual cost, reliance on specific operators/large language models (LLMs), and sparse reward signals. To address these challenges, we propose FlowSteer, an end-to-end reinforcement learning framework that takes a lightweight policy model as the agent and an executable canvas environment, automating workflow orchestration through multi-turn interaction. In this process, the policy model analyzes execution states and selects editing actions, while the canvas executes operators and returns feedback for iterative refinement. Moreover, FlowSteer provides a plug-and-play framework that supports diverse operator libraries and interchangeable LLM backends. To effectively train this interaction paradigm, we propose Canvas Workflow Relative Policy Optimization (CWRPO), which introduces diversity-constrained rewards with conditional release to stabilize learning and suppress shortcut behaviors. Experimental results on twelve datasets show that FlowSteer significantly outperforms baselines across various tasks.

翻译：近年来，各类强大的智能体工作流已被应用于解决广泛的人类问题。然而，现有工作流编排仍面临关键挑战，包括高昂的人工成本、对特定算子/大语言模型（LLMs）的依赖，以及稀疏的奖励信号。为应对这些挑战，我们提出FlowSteer——一种端到端的强化学习框架，该框架以轻量级策略模型作为智能体，并构建可执行的画布环境，通过多轮交互实现工作流编排的自动化。在此过程中，策略模型分析执行状态并选择编辑动作，而画布则执行算子并返回反馈以供迭代优化。此外，FlowSteer提供了一个即插即用的框架，支持多样化的算子库与可互换的LLM后端。为有效训练此交互范式，我们提出了画布工作流相对策略优化（CWRPO），该方法引入具有条件释放的多样性约束奖励，以稳定学习过程并抑制捷径行为。在十二个数据集上的实验结果表明，FlowSteer在多种任务上均显著优于基线方法。