It can be difficult to assess the quality of a fitted model when facing unsupervised learning problems. Latent variable models, such as variation autoencoders and Gaussian mixture models, are often trained with likelihood-based approaches. In scope of Goodhart's law, when a metric becomes a target it ceases to be a good metric and therefore we should not use likelihood to assess the quality of the fit of these models. The solution we propose is a new metric for model comparison or regularization that relies on moments. The concept is to study the difference between the data moments and the model moments using a matrix norm, such as the Frobenius norm. We show how to use this new metric for model comparison and then for regularization. It is common to draw samples from the fitted distribution when evaluating latent variable models and we show that our proposed metric is faster to compute and has a smaller variance that this alternative. We conclude this article with a proof of concept of both applications and we discuss future work.

翻译：在无监督学习问题中，评估拟合模型的质量可能具有挑战性。诸如变分自编码器和高斯混合模型等潜变量模型通常采用基于似然的方法进行训练。根据古德哈特定律，当一个指标成为目标时，它就不再是一个好的指标，因此我们不应使用似然来评估这些模型的拟合质量。我们提出的解决方案是一种新的度量，它依赖于矩，可用于模型比较或正则化。其核心思想是利用矩阵范数（如Frobenius范数）研究数据矩与模型矩之间的差异。我们展示了如何将此新度量用于模型比较，进而用于正则化。在评估潜变量模型时，通常从拟合分布中抽取样本，而我们提出的指标计算速度更快且方差更小。本文以概念验证结束，对这两种应用进行了实证，并讨论了未来工作方向。