Finding the optimal model complexity that minimizes the generalization error (GE) is a key issue of machine learning. For the conventional supervised learning, this task typically involves the bias-variance tradeoff: lowering the bias by making the model more complex entails an increase in the variance. Meanwhile, little has been studied about whether the same tradeoff exists for unsupervised learning. In this study, we propose that unsupervised learning generally exhibits a two-component tradeoff of the GE, namely the model error and the data error -- using a more complex model reduces the model error at the cost of the data error, with the data error playing a more significant role for a smaller training dataset. This is corroborated by training the restricted Boltzmann machine to generate the configurations of the two-dimensional Ising model at a given temperature and the totally asymmetric simple exclusion process with given entry and exit rates. Our results also indicate that the optimal model tends to be more complex when the data to be learned are more complex.

翻译：寻找最小化泛化误差（GE）的最优模型复杂度是机器学习的关键问题。对于传统的监督学习，这一任务通常涉及偏差-方差权衡：通过增加模型复杂度来降低偏差，会导致方差增大。然而，关于无监督学习是否存在类似的权衡鲜有研究。在本研究中，我们提出无监督学习通常呈现GE的两分量权衡，即模型误差和数据误差——使用更复杂的模型能以数据误差为代价降低模型误差，且训练数据集越小，数据误差的作用越显著。这一结论通过训练受限玻尔兹曼机生成给定温度下二维伊辛模型及给定出入速率下完全不对称简单排除过程的构型得到验证。我们的结果还表明，当待学习数据更复杂时，最优模型倾向于更复杂。