The maximum likelihood method is the best-known method for estimating the probabilities behind the data. However, the conventional method obtains the probability model closest to the empirical distribution, resulting in overfitting. Then regularization methods prevent the model from being excessively close to the wrong probability, but little is known systematically about their performance. The idea of regularization is similar to error-correcting codes, which obtain optimal decoding by mixing suboptimal solutions with an incorrectly received code. The optimal decoding in error-correcting codes is achieved based on gauge symmetry. We propose a theoretically guaranteed regularization in the maximum likelihood method by focusing on a gauge symmetry in Kullback -- Leibler divergence. In our approach, we obtain the optimal model without the need to search for hyperparameters frequently appearing in regularization.

翻译：最大似然法是估计数据背后概率最著名的方法。然而，传统方法获得最接近经验分布的概率模型，会导致过拟合。正则化方法虽能防止模型过度拟合错误概率，但其性能的系统性理解仍很有限。正则化的思想类似于纠错码，即通过将次优解与错误接收的码字混合来实现最优解码。纠错码中的最优解码基于规范对称性实现。我们通过聚焦库尔贝克–莱布勒散度中的规范对称性，提出了一种具有理论保证的最大似然法正则化方法。在我们的方法中，无需搜索正则化中频繁出现的超参数即可获得最优模型。