Finding the optimal model complexity that minimizes the generalization error (GE) is a key issue of machine learning. For the conventional supervised learning, this task typically involves the bias-variance tradeoff: lowering the bias by making the model more complex entails an increase in the variance. Meanwhile, little has been studied about whether the same tradeoff exists for unsupervised learning. In this study, we propose that unsupervised learning generally exhibits a two-component tradeoff of the GE, namely the model error and the data error -- using a more complex model reduces the model error at the cost of the data error, with the data error playing a more significant role for a smaller training dataset. This is corroborated by training the restricted Boltzmann machine to generate the configurations of the two-dimensional Ising model at a given temperature and the totally asymmetric simple exclusion process with given entry and exit rates. Our results also indicate that the optimal model tends to be more complex when the data to be learned are more complex.

翻译：寻找最小化泛化误差的最优模型复杂度是机器学习的关键问题。对于传统的监督学习，该任务通常涉及偏差-方差权衡：通过增加模型复杂度来降低偏差会导致方差的增加。然而，关于无监督学习是否存在相同的权衡，相关研究较少。本研究提出，无监督学习通常表现出泛化误差的两组分权衡，即模型误差和数据误差——使用更复杂的模型会以数据误差为代价降低模型误差，且当训练数据集较小时数据误差的作用更为显著。通过训练受限玻尔兹曼机生成给定温度下二维伊辛模型的构型以及具有给定入口和出口率的全不对称简单排斥过程，这一点得到了验证。我们的结果还表明，当需要学习的数据更复杂时，最优模型往往也更为复杂。