We study theoretical properties of a broad class of regularized algorithms with vector-valued output. These spectral algorithms include kernel ridge regression, kernel principal component regression, various implementations of gradient descent and many more. Our contributions are twofold. First, we rigorously confirm the so-called saturation effect for ridge regression with vector-valued output by deriving a novel lower bound on learning rates; this bound is shown to be suboptimal when the smoothness of the regression function exceeds a certain level. Second, we present the upper bound for the finite sample risk general vector-valued spectral algorithms, applicable to both well-specified and misspecified scenarios (where the true regression function lies outside of the hypothesis space) which is minimax optimal in various regimes. All of our results explicitly allow the case of infinite-dimensional output variables, proving consistency of recent practical applications.

翻译：我们研究了一类具有向量值输出的正则化算法的理论性质。这些谱算法包括核岭回归、核主成分回归、梯度下降的多种实现方式以及其他多种算法。我们的贡献主要有两方面。首先，通过推导学习速率的新下界，我们严格证实了向量值输出岭回归中所谓的饱和效应；该下界表明，当回归函数的平滑度超过特定水平时，该界是次优的。其次，我们给出了广义向量值谱算法有限样本风险的上界，该上界适用于设定正确和设定错误（真实回归函数位于假设空间之外）两种情形，并在多种机制下达到极小极大最优。我们所有的结果都明确允许输出变量为无限维的情形，从而证明了近期实际应用的一致性。