The maximum likelihood method is the best-known method for estimating the probabilities behind the data. However, the conventional method obtains the probability model closest to the empirical distribution, resulting in overfitting. Then regularization methods prevent the model from being excessively close to the wrong probability, but little is known systematically about their performance. The idea of regularization is similar to error-correcting codes, which obtain optimal decoding by mixing suboptimal solutions with an incorrectly received code. The optimal decoding in error-correcting codes is achieved based on gauge symmetry. We propose a theoretically guaranteed regularization in the maximum likelihood method by focusing on a gauge symmetry in Kullback -- Leibler divergence. In our approach, we obtain the optimal model without the need to search for hyperparameters frequently appearing in regularization.

翻译：最大似然方法是估计数据背后概率的最著名方法。然而，传统方法得到的是最接近经验分布的概率模型，导致过拟合。正则化方法虽能防止模型过度拟合错误概率，但其性能缺乏系统性理解。正则化的思想类似于纠错码——通过将次优解与错误接收的码字混合来获得最优解码。纠错码中的最优解码基于规范对称性实现。我们通过关注Kullback—Leibler散度中的规范对称性，提出了最大似然方法中具有理论保证的正则化方法。我们的方法无需搜索正则化中频繁出现的超参数即可获得最优模型。