Existing generative models, such as diffusion and auto-regressive networks, are inherently static, relying on a fixed set of pretrained parameters to handle all inputs. In contrast, humans flexibly adapt their internal generative representations to each perceptual or imaginative context. Inspired by this capability, we introduce Composer, a new paradigm for adaptive generative modeling based on test-time instance-specific parameter composition. Composer generates input-conditioned parameter adaptations at inference time, which are injected into the pretrained model's weights, enabling per-input specialization without fine-tuning or retraining. Adaptation occurs once prior to multi-step generation, yielding higher-quality, context-aware outputs with minimal computational and memory overhead. Experiments show that Composer substantially improves performance across diverse generative models and use cases, including lightweight/quantized models and test-time scaling. By leveraging input-aware parameter composition, Composer establishes a new paradigm for designing generative models that dynamically adapt to each input, moving beyond static parameterization.
翻译:现有生成模型(如扩散网络和自回归网络)本质上是静态的,依赖一组固定的预训练参数来处理所有输入。相比之下,人类能够灵活地调整其内部生成表征以适应每个感知或想象情境。受此能力启发,我们提出了Composer——一种基于测试时实例特定参数组合的自适应生成建模新范式。Composer在推理阶段生成输入条件化的参数适应项,并将其注入预训练模型的权重中,从而无需微调或重新训练即可实现针对每个输入的特化。适应过程在生成步骤开始前仅执行一次,以极小的计算和内存开销生成更高质量且具有上下文感知能力的输出。实验表明,Composer在多种生成模型和应用场景(包括轻量/量化模型及测试时缩放)中显著提升了性能。通过利用输入感知参数组合,Composer建立了一种新型生成模型设计范式,使模型能够动态适应每个输入,从而突破了静态参数化的局限。