Unified and scalable Transformers have recently achieved remarkable success in modeling diverse phenomena traditionally associated with computer graphics, such as 3D visual effects, rendering processes, and motion in videos. In this work, we take a step further by investigating whether modern Transformer techniques can tackle the challenging task of cloth simulation. To this end, we present ClothTransformer, a framework that reformulates cloth simulation as autoregressive sequence modeling in a learned latent space. Existing neural cloth simulators are largely specialized to single scenarios, intrinsically coupled to the mesh discretization, and lack robust collision handling. Our approach addresses these limitations through three contributions: (1) a unified Transformer architecture that handles diverse scenarios -- body-driven garments, robotic manipulation, and free-fall collisions -- under a single model and achieves approximately $4$--$9{\times}$ lower error than prior state-of-the-art methods across all scenarios; (2) a scalable latent-space formulation that compresses arbitrary-resolution meshes into a fixed-size set of latent tokens, making temporal dynamics computation independent of mesh resolution; and (3) a diverse-scenario high-fidelity penetration-free dataset of ${\sim}$493.4k frames spanning all three settings, which enables a differentiable Continuous Collision Detection (CCD) module to suppress penetration artifacts. Project Page: https://yucrazing.github.io/clothtransformer/

翻译：统一的、可扩展的Transformer近期在建模计算机图形学中多种传统现象（如三维视觉效果、渲染过程及视频中的运动）方面取得了显著成功。本文进一步探究现代Transformer技术能否解决布料模拟这一挑战性任务。为此，我们提出ClothTransformer框架，将布料模拟重构为在可学习潜在空间中的自回归序列建模。现有神经布料模拟器大多专用于单一场景，与网格离散化固有耦合，且缺乏稳健的碰撞处理。我们的方法通过三项贡献解决这些局限：（1）统一的Transformer架构，在单一模型下处理多样化场景——人体驱动服装、机器人操作及自由落体碰撞——并在所有场景中将误差降低至先前最先进方法的约4-9倍；（2）可扩展的潜在空间公式，将任意分辨率网格压缩为固定大小的潜在令牌集，使时域动力学计算与网格分辨率无关；（3）一个覆盖全部三种场景的高保真无穿透数据集（约49.34万帧），支持可微分的连续碰撞检测模块以抑制穿透伪影。项目页面：https://yucrazing.github.io/clothtransformer/