Recent work in Video Frame Interpolation (VFI) tries to formulate VFI as a diffusion-based conditional image generation problem, synthesizing the intermediate frame given a random noise and neighboring frames. Due to the relatively high resolution of videos, Latent Diffusion Models (LDMs) are employed as the conditional generation model, where the autoencoder compresses images into latent representations for diffusion and then reconstructs images from these latent representations. Such a formulation poses a crucial challenge: VFI expects that the output is deterministically equal to the ground truth intermediate frame, but LDMs randomly generate a diverse set of different images when the model runs multiple times. The reason for the diverse generation is that the cumulative variance (variance accumulated at each step of generation) of generated latent representations in LDMs is large. This makes the sampling trajectory random, resulting in diverse rather than deterministic generations. To address this problem, we propose our unique solution: Frame Interpolation with Consecutive Brownian Bridge Diffusion. Specifically, we propose consecutive Brownian Bridge diffusion that takes a deterministic initial value as input, resulting in a much smaller cumulative variance of generated latent representations. Our experiments suggest that our method can improve together with the improvement of the autoencoder and achieve state-of-the-art performance in VFI, leaving strong potential for further enhancement.

翻译：视频帧插值（VFI）的最新研究尝试将VFI表述为基于扩散的条件图像生成问题，即从随机噪声和相邻帧中合成中间帧。由于视频分辨率相对较高，采用潜在扩散模型（Latent Diffusion Models，LDMs）作为条件生成模型，其中自编码器将图像压缩为潜在表示以进行扩散，随后从这些潜在表示中重建图像。这种表述方式带来一个关键挑战：VFI期望输出结果确定性地等于真实中间帧，但LDMs在执行多次时会随机生成一组不同的图像。多样生成的原因在于LDMs中生成的潜在表示的累积方差（生成过程中每一步积累的方差）较大，导致采样轨迹随机化，从而产生多样化而非确定性的生成。为解决此问题，我们提出独特方案：利用连续布朗桥扩散的帧插值（Frame Interpolation with Consecutive Brownian Bridge Diffusion）。具体而言，我们提出连续布朗桥扩散方法，其输入采用确定性初值，从而使得生成的潜在表示的累积方差显著减小。实验结果表明，我们的方法能够随自编码器性能提升而同步改进，并在VFI任务中达到最优性能，为后续优化留有充足潜力。