Interpretable representation learning has been playing a key role in creative intelligent systems. In the music domain, current learning algorithms can successfully learn various features such as pitch, timbre, chord, texture, etc. However, most methods rely heavily on music domain knowledge. It remains an open question what general computational principles give rise to interpretable representations, especially low-dim factors that agree with human perception. In this study, we take inspiration from modern physics and use physical symmetry as a self-consistency constraint for the latent space. Specifically, it requires the prior model that characterises the dynamics of the latent states to be equivariant with respect to certain group transformations. We show that physical symmetry leads the model to learn a linear pitch factor from unlabelled monophonic music audio in a self-supervised fashion. In addition, the same methodology can be applied to computer vision, learning a 3D Cartesian space from videos of a simple moving object without labels. Furthermore, physical symmetry naturally leads to representation augmentation, a new technique which improves sample efficiency.

翻译：可解释表示学习在创造性智能系统中一直发挥着关键作用。在音乐领域，当前的学习算法能够成功学习音高、音色、和弦、织体等各种特征。然而，大多数方法严重依赖音乐领域知识。何种通用计算原理能够产生可解释的表示，尤其是与人类感知相符的低维因子，仍然是一个开放性问题。在本研究中，我们从现代物理学中汲取灵感，将物理对称性作为潜在空间的自洽性约束。具体而言，这要求描述潜在状态动态的先验模型在特定群变换下具有等变性。我们证明，物理对称性能够使模型以自监督方式从未标注的单音音乐音频中学习线性音高因子。此外，同样的方法可应用于计算机视觉，从未标注的简单移动物体视频中学习三维笛卡尔空间。更进一步，物理对称性自然引出了表示增强这一新技术，能够提升样本效率。