Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and geometrical attributes of neural representations as population-level mechanistic descriptors of task implementation. In particular, manifold capacity has emerged as a promising framework linking population geometry to the separability of neural manifolds. However, this metric has been limited to linear readouts. Here, we propose a theoretical framework that overcomes this limitation by leveraging contextual input information. We derive an exact formula for the context-dependent capacity that depends on manifold geometry and context correlations, and validate it on synthetic and real data. Our framework's increased expressivity captures representation untanglement in deep networks at early stages of the layer hierarchy, previously inaccessible to analysis. As context-dependent nonlinearity is ubiquitous in neural systems, our data-driven and theoretically grounded approach promises to elucidate context-dependent computation across scales, datasets, and models.

翻译：理解神经系统如何通过分布式表征高效处理信息，是神经科学与机器学习交叉领域的基础性挑战。近期研究将神经表征的统计与几何属性视为任务执行机制的人口级描述符。其中，流形容量作为将群体几何结构与神经流形可分离性相关联的框架崭露头角。然而，该指标此前仅局限于线性读出。本文提出一种利用上下文输入信息突破此限制的理论框架。我们推导了依赖于流形几何结构与上下文相关性的上下文依赖容量的精确公式，并在合成数据与真实数据上进行了验证。该框架增强的表达能力可捕获深度网络层级结构早期阶段先前无法分析的表征解缠现象。鉴于上下文依赖的非线性在神经系统中普遍存在，这种数据驱动且具有理论根基的方法有望阐明跨尺度、跨数据集与跨模型的上下文依赖计算机制。