We suggest a new multi-modal algorithm for joint inference of paired structurally aligned samples with Rectified Flow models. While some existing methods propose a codependent generation process, they do not view the problem of joint generation from a structural alignment perspective. Recent work uses Score Distillation Sampling to generate aligned 3D models, but SDS is known to be time-consuming, prone to mode collapse, and often provides cartoonish results. By contrast, our suggested approach relies on the joint transport of a segment in the sample space, yielding faster computation at inference time. Our approach can be built on top of an arbitrary Rectified Flow model operating on the structured latent space. We show the applicability of our method to the domains of image, video, and 3D shape generation using state-of-the-art baselines and evaluate it against both editing-based and joint inference-based competing approaches. We demonstrate a high degree of structural alignment for the sample pairs obtained with our method and a high visual quality of the samples. Our method improves the state-of-the-art for image and video generation pipelines. For 3D generation, it is able to show comparable quality while working orders of magnitude faster.

翻译：我们提出了一种基于整流流模型的新型多模态算法，用于联合推断成对结构对齐样本。现有方法虽提出了协同依赖的生成过程，但并未从结构对齐的视角审视联合生成问题。近期研究采用分数蒸馏采样技术生成对齐的3D模型，但该方法存在耗时严重、易陷入模式塌缩且常产生卡通化结果的缺陷。相比之下，我们提出的方法依赖于样本空间中片段的联合传输机制，在推断阶段可实现更高效的计算。本方法可构建于任意在结构化隐空间上运行的整流流模型之上。通过采用最先进的基线模型，我们展示了该方法在图像、视频及三维形状生成领域的适用性，并与基于编辑和基于联合推断的竞争方法进行了对比评估。实验表明，通过本方法获得的样本对具有高度的结构对齐特性，且样本视觉质量优异。本方法显著提升了图像与视频生成流程的技术水平，在三维生成领域则能在保持相当质量的同时实现数量级的速度提升。