Achieving efficient and robust multi-channel data learning is a challenging task in data science. By exploiting low-rankness in the transformed domain, i.e., transformed low-rankness, tensor Singular Value Decomposition (t-SVD) has achieved extensive success in multi-channel data representation and has recently been extended to function representation such as Neural Networks with t-product layers (t-NNs). However, it still remains unclear how t-SVD theoretically affects the learning behavior of t-NNs. This paper is the first to answer this question by deriving the upper bounds of the generalization error of both standard and adversarially trained t-NNs. It reveals that the t-NNs compressed by exact transformed low-rank parameterization can achieve a sharper adversarial generalization bound. In practice, although t-NNs rarely have exactly transformed low-rank weights, our analysis further shows that by adversarial training with gradient flow (GF), the over-parameterized t-NNs with ReLU activations are trained with implicit regularization towards transformed low-rank parameterization under certain conditions. We also establish adversarial generalization bounds for t-NNs with approximately transformed low-rank weights. Our analysis indicates that the transformed low-rank parameterization can promisingly enhance robust generalization for t-NNs.

翻译：实现高效且鲁棒的多通道数据学习是数据科学领域的一项挑战性任务。通过利用变换域中的低秩性（即变换低秩性），张量奇异值分解（t-SVD）已在多通道数据表征方面取得广泛成功，并近期被扩展至函数表征领域，例如具有t-积层的神经网络（t-NNs）。然而，t-SVD如何从理论上影响t-NNs的学习行为仍不明确。本文首次通过推导标准训练与对抗训练下t-NNs泛化误差的上界来回答这一问题。研究表明，通过精确变换低秩参数化压缩的t-NNs能够获得更紧的对抗泛化界。在实际场景中，尽管t-NNs很少具有精确的变换低秩权重，但我们的分析进一步表明：在梯度流（GF）对抗训练过程中，具有ReLU激活函数的过参数化t-NNs会在特定条件下被训练为隐式正则化趋向于变换低秩参数化。我们还为具有近似变换低秩权重的t-NNs建立了对抗泛化界。分析表明，变换低秩参数化有望显著增强t-NNs的鲁棒泛化能力。