Working memory is a central cognitive ability crucial for intelligent decision-making. Recent experimental and computational work studying working memory has primarily used categorical (i.e., one-hot) inputs, rather than ecologically relevant, multidimensional naturalistic ones. Moreover, studies have primarily investigated working memory during single or few cognitive tasks. As a result, an understanding of how naturalistic object information is maintained in working memory in neural networks is still lacking. To bridge this gap, we developed sensory-cognitive models, comprising a convolutional neural network (CNN) coupled with a recurrent neural network (RNN), and trained them on nine distinct N-back tasks using naturalistic stimuli. By examining the RNN's latent space, we found that: (1) Multi-task RNNs represent both task-relevant and irrelevant information simultaneously while performing tasks; (2) The latent subspaces used to maintain specific object properties in vanilla RNNs are largely shared across tasks, but highly task-specific in gated RNNs such as GRU and LSTM; (3) Surprisingly, RNNs embed objects in new representational spaces in which individual object features are less orthogonalized relative to the perceptual space; (4) The transformation of working memory encodings (i.e., embedding of visual inputs in the RNN latent space) into memory was shared across stimuli, yet the transformations governing the retention of a memory in the face of incoming distractor stimuli were distinct across time. Our findings indicate that goal-driven RNNs employ chronological memory subspaces to track information over short time spans, enabling testable predictions with neural data.

翻译：工作记忆是一种核心认知能力，对智能决策至关重要。近期关于工作记忆的实验与计算研究主要采用分类（即独热编码）输入，而非生态相关的多维自然刺激。此外，现有研究大多仅在单一或少数认知任务中考察工作记忆机制。因此，关于自然物体信息如何在神经网络的工作记忆中得到保持，目前仍缺乏深入理解。为填补这一空白，我们开发了感觉-认知耦合模型，该模型由卷积神经网络（CNN）与循环神经网络（RNN）级联构成，并使用自然刺激在九种不同的N-back任务上对其进行训练。通过对RNN潜在空间的解析，我们发现：（1）多任务RNN在执行任务时能同时表征任务相关与无关信息；（2）在普通RNN中，用于保持特定物体属性的潜在子空间在不同任务间高度共享，而在门控RNN（如GRU和LSTM）中则表现出强烈的任务特异性；（3）令人惊讶的是，RNN将物体嵌入到新的表征空间中，其中个体物体特征相对于感知空间的正交性显著降低；（4）工作记忆编码（即视觉输入在RNN潜在空间中的嵌入）向记忆的转换过程在不同刺激间具有共性，然而在面对干扰刺激时维持记忆的转换机制随时间呈现特异性。我们的研究结果表明，目标驱动的RNN通过时序记忆子空间追踪短时程信息，这为神经数据检验提供了可验证的预测框架。