Neural networks transform high-dimensional data into compact, structured representations, often modeled as elements of a lower dimensional latent space. In this paper, we present an alternative interpretation of neural models as dynamical systems acting on the latent manifold. Specifically, we show that autoencoder models implicitly define a latent vector field on the manifold, derived by iteratively applying the encoding-decoding map, without any additional training. We observe that standard training procedures introduce inductive biases that lead to the emergence of attractor points within this vector field. Drawing on this insight, we propose to leverage the vector field as a representation for the network, providing a novel tool to analyze the properties of the model and the data. This representation enables to: (i) analyze the generalization and memorization regimes of neural models, even throughout training; (ii) extract prior knowledge encoded in the network's parameters from the attractors, without requiring any input data; (iii) identify out-of-distribution samples from their trajectories in the vector field. We further validate our approach on vision foundation models, showcasing the applicability and effectiveness of our method in real-world scenarios.

翻译：神经网络将高维数据转化为紧凑的结构化表示，通常建模为低维潜在空间中的元素。本文提出了一种替代解释，将神经模型视为作用于潜在流形的动力系统。具体而言，我们证明自编码器模型通过迭代应用编码-解码映射，隐式定义了流形上的潜在向量场，且无需任何额外训练。我们观察到标准训练过程引入了归纳偏差，导致该向量场中出现吸引子点。基于这一发现，我们提出利用向量场作为网络的表示，为分析模型和数据属性提供新的工具。该表示能够实现：（1）分析神经模型的泛化与记忆机制（甚至涵盖训练全过程）；（2）从吸引子中提取网络参数编码的先验知识，无需任何输入数据；（3）根据样本在向量场中的轨迹识别分布外样本。我们进一步在视觉基础模型上验证了该方法，展示了其在实际场景中的适用性和有效性。