We study offline reinforcement learning of style-conditioned policies using explicit style supervision via subtrajectory labeling functions. In this setting, aligning style with high task performance is particularly challenging due to distribution shift and inherent conflicts between style and reward. Existing methods, despite introducing numerous definitions of style, often fail to reconcile these objectives effectively. To address these challenges, we propose a unified definition of behavior style and instantiate it into a practical framework. Building on this, we introduce Style-Conditioned Implicit Q-Learning (SCIQL), which leverages offline goal-conditioned RL techniques, such as hindsight relabeling and value learning, and combine it with a new Gated Advantage Weighted Regression mechanism to efficiently optimize task performance while preserving style alignment. Experiments demonstrate that SCIQL achieves superior performance on both objectives compared to prior offline methods. Code, datasets and visuals are available in: https://sciql-iclr-2026.github.io/.

翻译：本研究通过子轨迹标注函数提供的显式风格监督，探索风格条件化策略的离线强化学习方法。在此设定下，由于分布偏移以及风格与奖励之间的固有冲突，实现风格与高任务性能的对齐尤为困难。现有方法尽管引入了多种风格定义，却往往难以有效协调这两个目标。为应对这些挑战，我们提出了行为风格的统一定义，并将其实例化为一个实用框架。在此基础上，我们提出了风格条件化隐式Q学习（SCIQL），该方法利用离线目标条件强化学习技术（如后见之重新标注与价值学习），并结合新型门控优势加权回归机制，以在保持风格对齐的同时高效优化任务性能。实验表明，与先前的离线方法相比，SCIQL在两项目标上均实现了更优的性能。代码、数据集及可视化结果发布于：https://sciql-iclr-2026.github.io/。