We evaluate JEPA-style predictive representation learning versus reconstruction-based autoencoders on a controlled "TV-series" linear dynamical system with known latent state and a single noise parameter. While an initial comparison suggests JEPA is markedly more robust to noise, further diagnostics show that autoencoder failures are strongly influenced by asymmetries in objectives and by bottleneck/component-selection effects (confirmed by PCA baselines). Motivated by these findings, we introduce gated predictive autoencoders that learn to select predictable components, mimicking the beneficial feature-selection behavior observed in over-parameterized PCA. On this toy testbed, the proposed gated model is stable across noise levels and matches or outperforms JEPA.

翻译：我们在一个受控的"电视连续剧"线性动态系统上评估了JEPA风格的预测表征学习与基于重构的自编码器，该系统具有已知的潜状态和单一噪声参数。初步比较表明JEPA对噪声具有显著更强的鲁棒性，但进一步的诊断分析显示，自编码器的失败主要受目标函数不对称性以及瓶颈/组件选择效应的影响（这通过PCA基线得到了证实）。受这些发现启发，我们提出了门控预测自编码器，该模型能够学习选择可预测的组件，模拟了在过参数化PCA中观察到的有益特征选择行为。在此玩具测试平台上，所提出的门控模型在不同噪声水平下均保持稳定，其性能达到或超越了JEPA。