We present a constructive universal approximation theorem for learning machines equipped with joint-group-equivariant feature maps, called the joint-equivariant machines, based on the group representation theory. "Constructive" here indicates that the distribution of parameters is given in a closed-form expression known as the ridgelet transform. Joint-group-equivariance encompasses a broad class of feature maps that generalize classical group-equivariance. Particularly, fully-connected networks are not group-equivariant but are joint-group-equivariant. Our main theorem also unifies the universal approximation theorems for both shallow and deep networks. Until this study, the universality of deep networks has been shown in a different manner from the universality of shallow networks, but our results discuss them on common ground. Now we can understand the approximation schemes of various learning machines in a unified manner. As applications, we show the constructive universal approximation properties of four examples: depth-$n$ joint-equivariant machine, depth-$n$ fully-connected network, depth-$n$ group-convolutional network, and a new depth-$2$ network with quadratic forms whose universality has not been known.

翻译：基于群表示理论，我们为配备联合群等变特征映射（称为联合等变机器）的学习机器提出了一个构造性通用逼近定理。此处的“构造性”意指参数分布以闭合形式表达式——即脊波变换——给出。联合群等变性涵盖了一类广泛的特征映射，其推广了经典的群等变性。特别地，全连接网络虽非群等变，但属于联合群等变。我们的主定理还统一了浅层与深层网络的通用逼近定理。在本研究之前，深层网络的普适性一直以不同于浅层网络普适性的方式被证明，但我们的结果在共同基础上讨论了二者。现在我们可以统一地理解各种学习机器的逼近方案。作为应用，我们展示了四个示例的构造性通用逼近性质：深度-$n$ 联合等变机器、深度-$n$ 全连接网络、深度-$n$ 群卷积网络，以及一种新的深度-$2$ 二次型网络（其普适性此前未知）。