Achieving efficient and robust multi-channel data learning is a challenging task in data science. By exploiting low-rankness in the transformed domain, i.e., transformed low-rankness, tensor Singular Value Decomposition (t-SVD) has achieved extensive success in multi-channel data representation and has recently been extended to function representation such as Neural Networks with t-product layers (t-NNs). However, it still remains unclear how t-SVD theoretically affects the learning behavior of t-NNs. This paper is the first to answer this question by deriving the upper bounds of the generalization error of both standard and adversarially trained t-NNs. It reveals that the t-NNs compressed by exact transformed low-rank parameterization can achieve a sharper adversarial generalization bound. In practice, although t-NNs rarely have exactly transformed low-rank weights, our analysis further shows that by adversarial training with gradient flow (GF), the over-parameterized t-NNs with ReLU activations are trained with implicit regularization towards transformed low-rank parameterization under certain conditions. We also establish adversarial generalization bounds for t-NNs with approximately transformed low-rank weights. Our analysis indicates that the transformed low-rank parameterization can promisingly enhance robust generalization for t-NNs.

翻译：实现高效鲁棒的多通道数据学习是数据科学中的一项挑战性任务。通过利用变换域的低秩性（即变换低秩性），张量奇异值分解（t-SVD）在多通道数据表示中取得了广泛成功，并近期被扩展至函数表示领域，例如带有t-乘积层的神经网络（t-NNs）。然而，t-SVD如何从理论上影响t-NNs的学习行为仍不清楚。本文首次通过推导标准训练与对抗训练下t-NNs的泛化误差上界来回答这一问题。研究表明，通过精确变换低秩参数化压缩的t-NNs能够获得更优的对抗泛化界。在实践中，尽管t-NNs很少具有精确的变换低秩权重，我们的分析进一步表明，通过梯度流（GF）进行对抗训练时，在特定条件下，使用ReLU激活的过参数化t-NNs会隐式正则化趋向于变换低秩参数化。我们还为具有近似变换低秩权重的t-NNs建立了对抗泛化界。分析表明，变换低秩参数化能够有效提升t-NNs的鲁棒泛化能力。