To endow models with greater understanding of physics and motion, it is useful to enable them to perceive how solid surfaces move and deform in real scenes. This can be formalized as Tracking-Any-Point (TAP), which requires the algorithm to track any point on solid surfaces in a video, potentially densely in space and time. Large-scale groundtruth training data for TAP is only available in simulation, which currently has a limited variety of objects and motion. In this work, we demonstrate how large-scale, unlabeled, uncurated real-world data can improve a TAP model with minimal architectural changes, using a selfsupervised student-teacher setup. We demonstrate state-of-the-art performance on the TAP-Vid benchmark surpassing previous results by a wide margin: for example, TAP-Vid-DAVIS performance improves from 61.3% to 67.4%, and TAP-Vid-Kinetics from 57.2% to 62.5%. For visualizations, see our project webpage at https://bootstap.github.io/

翻译：为使模型具备更强的物理与运动理解能力，提升其对真实场景中固体表面运动与形变的感知能力至关重要。这一问题可形式化为任意点跟踪（Tracking-Any-Point, TAP），其要求算法能够追踪视频中固体表面的任意点，并可能在时空维度实现密集追踪。目前大规模TAP真实标注训练数据仅能通过仿真获取，而现有仿真数据在物体种类与运动模式上存在局限。本研究通过自监督师生架构，在最小化结构改动的前提下，论证如何利用大规模、未标注、未筛选的真实世界数据提升TAP模型性能。我们在TAP-Vid基准测试中实现了最先进的性能表现，较先前结果有显著提升：例如TAP-Vid-DAVIS的准确率从61.3%提升至67.4%，TAP-Vid-Kinetics从57.2%提升至62.5%。可视化结果请参见项目网页 https://bootstap.github.io/