Current Dynamic Texture Synthesis (DyTS) models can synthesize realistic videos. However, they require a slow iterative optimization process to synthesize a single fixed-size short video, and they do not offer any post-training control over the synthesis process. We propose Dynamic Neural Cellular Automata (DyNCA), a framework for real-time and controllable dynamic texture synthesis. Our method is built upon the recently introduced NCA models and can synthesize infinitely long and arbitrary-sized realistic video textures in real time. We quantitatively and qualitatively evaluate our model and show that our synthesized videos appear more realistic than the existing results. We improve the SOTA DyTS performance by $2\sim 4$ orders of magnitude. Moreover, our model offers several real-time video controls including motion speed, motion direction, and an editing brush tool. We exhibit our trained models in an online interactive demo that runs on local hardware and is accessible on personal computers and smartphones.

翻译：现有动态纹理合成模型能够生成逼真的视频，但需经历缓慢的迭代优化过程才能合成单一固定尺寸的短视频，且无法在训练后对合成过程进行控制。我们提出动态神经细胞自动机（DyNCA），这是一种实现实时可控动态纹理合成的框架。该方法基于近期提出的神经细胞自动机模型，可实时合成无限长度、任意尺寸的逼真视频纹理。通过定量与定性评估，我们合成的视频在真实感上优于现有成果，并将动态纹理合成性能提升2~4个数量级。此外，该模型支持多种实时视频控制功能，包括运动速度、运动方向调节以及编辑笔刷工具。我们在本地硬件运行的在线交互式演示中展示了训练好的模型，该演示可通过个人电脑和智能手机访问。