The Unconstrained Feature Model (UFM) is a mathematical framework that enables closed-form approximations for minimal training loss and related performance measures in deep neural networks (DNNs). This paper leverages the UFM to provide qualitative insights into neural multivariate regression, a critical task in imitation learning, robotics, and reinforcement learning. Specifically, we address two key questions: (1) How do multi-task models compare to multiple single-task models in terms of training performance? (2) Can whitening and normalizing regression targets improve training performance? The UFM theory predicts that multi-task models achieve strictly smaller training MSE than multiple single-task models when the same or stronger regularization is applied to the latter, and our empirical results confirm these findings. Regarding whitening and normalizing regression targets, the UFM theory predicts that they reduce training MSE when the average variance across the target dimensions is less than one, and our empirical results once again confirm these findings. These findings highlight the UFM as a powerful framework for deriving actionable insights into DNN design and data pre-processing strategies.

翻译：无约束特征模型（UFM）是一种数学框架，能够为深度神经网络（DNNs）中的最小训练损失及相关性能度量提供闭式近似。本文利用UFM为神经多元回归提供定性洞见，该任务是模仿学习、机器人学和强化学习中的关键任务。具体而言，我们探讨了两个核心问题：（1）多任务模型与多个单任务模型在训练性能上如何比较？（2）对回归目标进行白化和归一化是否能提升训练性能？UFM理论预测，当对多个单任务模型施加相同或更强的正则化时，多任务模型能获得严格更小的训练均方误差（MSE），我们的实证结果证实了这一预测。关于对回归目标进行白化和归一化，UFM理论预测，当目标维度间的平均方差小于1时，这些操作能降低训练MSE，我们的实证结果再次验证了这一预测。这些发现凸显了UFM作为一个强大框架，能够为DNN设计和数据预处理策略提供可操作的洞见。