A key property of neural networks (both biological and artificial) is how they learn to represent and manipulate input information in order to solve a task. Different types of representations may be suited to different types of tasks, making identifying and understanding learned representations a critical part of understanding and designing useful networks. In this paper, we introduce a new pseudo-kernel based tool for analyzing and predicting learned representations, based only on the initial conditions of the network and the training curriculum. We validate the method on a simple test case, before demonstrating its use on a question about the effects of representational learning on sequential single versus concurrent multitask performance. We show that our method can be used to predict the effects of the scale of weight initialization and training curriculum on representational learning and downstream concurrent multitasking performance.

翻译：神经网络（包括生物网络和人工网络）的一个关键特性是它们如何学习表征和操纵输入信息以解决特定任务。不同类型的表征可能适用于不同类型的任务，因此识别和理解学习到的表征对于理解与设计实用网络至关重要。本文基于网络的初始条件和训练课程，引入了一种新的伪核工具，用于分析和预测学习到的表征。我们首先在一个简单测试案例上验证该方法，随后将其应用于探究表征学习对顺序单任务与并发多任务性能影响的问题。结果表明，该方法可用于预测权重初始化尺度和训练课程对表征学习及下游并发多任务性能的影响。