Adapting large language models to multiple tasks can cause cross-skill interference, where improvements for one skill degrade another. While methods such as LoRA impose orthogonality constraints at the weight level, they do not fully address interference in hidden-state representations. We propose Compositional Subspace Representation Fine-tuning (CS-ReFT), a novel representation-based approach that learns multiple orthonormal subspace transformations, each specializing in a distinct skill, and composes them via a lightweight router. By isolating these subspace edits in the hidden state, rather than weight matrices, CS-ReFT prevents cross-task conflicts more effectively. On the AlpacaEval benchmark, applying CS-ReFT to Llama-2-7B achieves a 93.94% win rate, surpassing GPT-3.5 Turbo (86.30%) while requiring only 0.0098% of model parameters. These findings show that specialized representation edits, composed via a simple router, significantly enhance multi-task instruction following with minimal overhead.

翻译：大语言模型适应多任务时可能引发跨技能干扰，即某一技能提升导致其他技能退化。尽管LoRA等方法在权重层面施加正交性约束，但未能完全解决隐藏状态表示中的干扰问题。本文提出组合子空间表示微调（CS-ReFT），这是一种基于表示学习的新型方法，通过训练多个正交子空间变换（每个变换专精于特定技能），并借助轻量级路由器进行组合。通过在隐藏状态而非权重矩阵中隔离这些子空间编辑，CS-ReFT能更有效地防止跨任务冲突。在AlpacaEval基准测试中，将CS-ReFT应用于Llama-2-7B模型获得了93.94%的胜率，超越GPT-3.5 Turbo（86.30%），而所需参数量仅占模型的0.0098%。实验表明，通过简单路由器组合的专用表示编辑，能以极低开销显著提升多任务指令跟随能力。