We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on the group representation theory. "Constructive" here indicates that the distribution of parameters is given in a closed-form expression known as the ridgelet transform. Joint-group-equivariance encompasses a broad class of feature maps that generalize classical group-equivariance. Particularly, fully-connected networks are not group-equivariant but are joint-group-equivariant. Our main theorem also unifies the universal approximation theorems for both shallow and deep networks. Until this study, the universality of deep networks has been shown in a different manner from the universality of shallow networks, but our results discuss them on common ground. Now we can understand the approximation schemes of various learning machines in a unified manner. As applications, we show the constructive universal approximation properties of four examples: depth-$n$ joint-equivariant machine, depth-$n$ fully-connected network, depth-$n$ group-convolutional network, and a new depth-$2$ network with quadratic forms whose universality has not been known.

翻译：基于群表示理论，我们为配备联合群等变特征映射的学习机器（称为联合等变机器）提出了一个构造性通用逼近定理。此处的“构造性”指参数分布以闭式表达式——即脊波变换——给出。联合群等变性涵盖了一类广泛的特征映射，其推广了经典的群等变性。特别地，全连接网络虽非群等变，但属于联合群等变。我们的主定理同时统一了浅层与深层网络的通用逼近定理。在本研究之前，深层网络的普适性一直以不同于浅层网络的方式证明，而我们的结果在共同基础上讨论二者。现在我们可以统一理解各类学习机器的逼近机制。作为应用，我们展示了四个示例的构造性通用逼近性质：深度-$n$联合等变机器、深度-$n$全连接网络、深度-$n$群卷积网络，以及一种其普适性此前未知的、具有二次型的新型深度-$2$网络。