We introduce SuperIgor, a framework for instruction-following tasks. Unlike prior methods that rely on predefined subtasks, SuperIgor enables a language model to generate and refine high-level plans through a self-learning mechanism, reducing the need for manual dataset annotation. Our approach involves iterative co-training: an RL agent is trained to follow the generated plans, while the language model adapts and modifies these plans based on RL feedback and preferences. This creates a feedback loop where both the agent and the planner improve jointly. We validate our framework in environments with rich dynamics and stochasticity. Results show that SuperIgor agents adhere to instructions more strictly than baseline methods, while also demonstrating strong generalization to previously unseen instructions.

翻译：我们提出SuperIgor框架，一种面向指令跟随任务的系统方案。与依赖预定义子任务的现有方法不同，SuperIgor通过自学习机制使语言模型能够生成并优化高层计划，从而减少人工数据集标注需求。本方法采用迭代协同训练策略：强化学习代理学习执行生成的计划，而语言模型则根据强化学习反馈与偏好动态调整优化计划。这种双向反馈机制使代理与规划器能够协同进化。我们在具有丰富动态特性和随机性的环境中验证了该框架。实验结果表明，SuperIgor代理相比基线方法能更严格遵循指令，同时展现出对未见指令的强泛化能力。