Implicit Neural Representations (INRs) are nowadays used to represent multimedia signals across various real-life applications, including image super-resolution, image compression, or 3D rendering. Existing methods that leverage INRs are predominantly focused on visual data, as their application to other modalities, such as audio, is nontrivial due to the inductive biases present in architectural attributes of image-based INR models. To address this limitation, we introduce HyperSound, the first meta-learning approach to produce INRs for audio samples that leverages hypernetworks to generalize beyond samples observed in training. Our approach reconstructs audio samples with quality comparable to other state-of-the-art models and provides a viable alternative to contemporary sound representations used in deep neural networks for audio processing, such as spectrograms.
翻译:隐式神经表示(INR)如今被广泛应用于多种实际场景中的多媒体信号表示,包括图像超分辨率、图像压缩或三维渲染。现有利用INR的方法主要集中在视觉数据上,但由于基于图像的INR模型架构属性中存在归纳偏置,这些方法在音频等其他模态上的应用具有显著挑战。为解决这一局限,我们提出HyperSound——首个利用超网络为音频样本生成INR的元学习方法,该方法能够泛化到训练过程中未见过的样本。我们的方法能以与最先进模型相当的质量重建音频样本,并为当前深度神经网络中用于音频处理的现代声音表示(如频谱图)提供了一种可行的替代方案。