We suggest a new multi-modal algorithm for joint inference of paired structurally aligned samples with Rectified Flow models. While some existing methods propose a codependent generation process, they do not view the problem of joint generation from a structural alignment perspective. Recent work uses Score Distillation Sampling to generate aligned 3D models, but SDS is known to be time-consuming, prone to mode collapse, and often provides cartoonish results. By contrast, our suggested approach relies on the joint transport of a segment in the sample space, yielding faster computation at inference time. Our approach can be built on top of an arbitrary Rectified Flow model operating on the structured latent space. We show the applicability of our method to the domains of image, video, and 3D shape generation using state-of-the-art baselines and evaluate it against both editing-based and joint inference-based competing approaches. We demonstrate a high degree of structural alignment for the sample pairs obtained with our method and a high visual quality of the samples. Our method improves the state-of-the-art for image and video generation pipelines. For 3D generation, it is able to show comparable quality while working orders of magnitude faster.

翻译：我们提出了一种新型多模态算法，用于结合整流流模型对成对结构对齐样本进行联合推理。现有方法虽提出协同生成过程，但未从结构对齐视角看待联合生成问题。近期研究利用分数蒸馏采样生成对齐的三维模型，但SDS方法耗时、易出现模式坍塌且结果常呈现卡通化。相比之下，我们提出的方法基于样本空间中的片段联合传输，在推理时实现了更快的计算速度。该方法可构建于任意在结构化潜空间上运行的整流流模型之上。我们展示了该方法在图像、视频和三维形状生成领域的适用性，基于最先进基线模型，并与基于编辑和基于联合推理的竞争方法进行评估。实验结果表明，我们的方法在样本对结构对齐度与视觉质量方面均表现出色，提升了图像与视频生成管线的当前最优水平；在三维生成领域，该方法在达到可比质量的同时，运算速度快了数个数量级。