Representation multi-task learning (MTL) and transfer learning (TL) have achieved tremendous success in practice. However, the theoretical understanding of these methods is still lacking. Most existing theoretical works focus on cases where all tasks share the same representation, and claim that MTL and TL almost always improve performance. However, as the number of tasks grows, assuming all tasks share the same representation is unrealistic. Also, this does not always match empirical findings, which suggest that a shared representation may not necessarily improve single-task or target-only learning performance. In this paper, we aim to understand how to learn from tasks with \textit{similar but not exactly the same} linear representations, while dealing with outlier tasks. With a known intrinsic dimension, we propose two algorithms that are \textit{adaptive} to the similarity structure and \textit{robust} to outlier tasks under both MTL and TL settings. Our algorithms outperform single-task or target-only learning when representations across tasks are sufficiently similar and the fraction of outlier tasks is small. Furthermore, they always perform no worse than single-task learning or target-only learning, even when the representations are dissimilar. We provide information-theoretic lower bounds to show that our algorithms are nearly \textit{minimax} optimal in a large regime. We also propose an algorithm to adapt to the unknown intrinsic dimension. We conduct two simulation studies to verify our theoretical results.

翻译：多任务表示学习（MTL）和迁移学习（TL）在实践中取得了巨大成功。然而，这些方法的理论基础仍相对匮乏。现有理论工作大多聚焦于所有任务共享同一表示的情形，并声称MTL和TL几乎总能提升性能。但随着任务数量增加，假设所有任务共享相同表示并不现实。此外，这与实证发现并不完全一致——实证表明共享表示未必能提升单任务或仅目标任务的学习性能。本文旨在探究如何从具有**相似但不完全相同**的线性表示的任务中学习，同时处理异常任务。在已知内在维度的前提下，我们提出了两种算法，这些算法在MTL和TL设置下既能**自适应**于相似性结构，又能对异常任务具有**鲁棒性**。当任务间表示足够相似且异常任务占比小时，我们的算法优于单任务或仅目标任务学习；更关键的是，即使表示不相似，其性能也始终不低于单任务学习或仅目标任务学习。我们给出了信息论下界，证明所提算法在较大范围内接近**极小极大**最优性。此外，我们还提出了一种可自适应未知内在维度的算法。通过两项仿真研究验证了理论结果。