The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visual relationships in addition to the already existing challenges in image-based SGG. Existing methods for dynamic SGG have primarily focused on capturing spatio-temporal context using complex architectures without addressing the challenges mentioned above, especially the long-tailed distribution of relationships. This often leads to the generation of biased scene graphs. To address these challenges, we introduce a new framework called TEMPURA: TEmporal consistency and Memory Prototype guided UnceRtainty Attenuation for unbiased dynamic SGG. TEMPURA employs object-level temporal consistencies via transformer-based sequence modeling, learns to synthesize unbiased relationship representations using memory-guided training, and attenuates the predictive uncertainty of visual relations using a Gaussian Mixture Model (GMM). Extensive experiments demonstrate that our method achieves significant (up to 10% in some cases) performance gain over existing methods highlighting its superiority in generating more unbiased scene graphs.

翻译：动态场景图生成（SGG）任务因场景固有动态性、模型预测的时间波动性、视觉关系的长尾分布以及图像级SGG中已有的挑战而变得复杂且具有挑战性。现有动态SGG方法主要聚焦于利用复杂架构捕获时空上下文，却未能应对上述挑战（尤其是关系的长尾分布），导致常生成有偏场景图。为解决这些问题，我们提出了一种名为TEMPURA的新框架：基于时间一致性与记忆原型引导的不确定性衰减实现无偏动态SGG。TEMPURA通过基于Transformer的序列建模实现目标级时间一致性，利用记忆引导训练学习合成无偏关系表征，并借助高斯混合模型（GMM）衰减视觉关系的预测不确定性。大量实验表明，我们的方法相比现有方法实现了显著性能提升（部分场景高达10%），凸显了其在生成更无偏场景图方面的优越性。