In this work, we investigate the structure and representation capacity of sinusoidal MLPs - multilayer perceptron networks that use sine as the activation function. These neural networks (known as neural fields) have become fundamental in representing common signals in computer graphics, such as images, signed distance functions, and radiance fields. This success can be primarily attributed to two key properties of sinusoidal MLPs: smoothness and compactness. These functions are smooth because they arise from the composition of affine maps with the sine function. This work provides theoretical results to justify the compactness property of sinusoidal MLPs and provides control mechanisms in the definition and training of these networks. We propose to study a sinusoidal MLP by expanding it as a harmonic sum. First, we observe that its first layer can be seen as a harmonic dictionary, which we call the input sinusoidal neurons. Then, a hidden layer combines this dictionary using an affine map and modulates the outputs using the sine, this results in a special dictionary of sinusoidal neurons. We prove that each of these sinusoidal neurons expands as a harmonic sum producing a large number of new frequencies expressed as integer linear combinations of the input frequencies. Thus, each hidden neuron produces the same frequencies, and the corresponding amplitudes are completely determined by the hidden affine map. We also provide an upper bound and a way of sorting these amplitudes that can control the resulting approximation, allowing us to truncate the corresponding series. Finally, we present applications for training and initialization of sinusoidal MLPs. Additionally, we show that if the input neurons are periodic, then the entire network will be periodic with the same period. We relate these periodic networks with the Fourier series representation.

翻译：本文研究正弦MLP（使用正弦函数作为激活函数的多层感知器网络）的结构与表示能力。这类神经网络（称为神经场）已成为计算机图形学中表示常见信号（如图像、符号距离函数和辐射场）的基础工具。其成功主要归功于正弦MLP的两个关键特性：平滑性与紧凑性。这些函数之所以平滑，是因为它们由仿射映射与正弦函数复合而成。本文提供了理论结果以论证正弦MLP的紧凑性，并给出了此类网络定义与训练中的控制机制。我们通过将正弦MLP展开为调和级数来对其进行研究。首先，我们观察到其第一层可视为谐波字典，称为输入正弦神经元。随后，隐藏层通过仿射映射组合该字典，并利用正弦函数调节输出，从而形成特殊的正弦神经元字典。我们证明每个此类正弦神经元可展开为调和级数，生成大量新频率，这些频率可表示为输入频率的整数线性组合。因此，每个隐藏神经元产生相同频率，对应振幅完全由隐藏仿射映射决定。我们还提供了振幅的上界及排序方法，可控制最终逼近结果，从而允许截断相应级数。最后，我们提出了正弦MLP训练与初始化的应用方法。此外，我们证明若输入神经元具有周期性，则整个网络将具有相同周期。我们将此类周期网络与傅里叶级数表示联系起来。