Click-Through Rate (CTR) prediction has become an essential task in digital industries, such as digital advertising or online shopping. Many deep learning-based methods have been implemented and have become state-of-the-art models in the domain. To further improve the performance of CTR models, Knowledge Distillation based approaches have been widely used. However, most of the current CTR prediction models do not have much complex architectures, so it's hard to call one of them 'cumbersome' and the other one 'tiny'. On the other hand, the performance gap is also not very large between complex and simple models. So, distilling knowledge from one model to the other could not be worth the effort. Under these considerations, Mutual Learning could be a better approach, since all the models could be improved mutually. In this paper, we showed how useful the mutual learning algorithm could be when it is between equals. In our experiments on the Criteo and Avazu datasets, the mutual learning algorithm improved the performance of the model by up to 0.66% relative improvement.

翻译：点击率（CTR）预测已成为数字广告和在线购物等数字产业中的核心任务。众多基于深度学习的方法已被广泛应用，并成为该领域的先进模型。为进一步提升CTR模型的性能，基于知识蒸馏的方法得到了广泛采用。然而，当前大多数CTR预测模型并未采用过于复杂的架构，因此难以将其区分为“笨重”模型与“轻量”模型。此外，复杂模型与简单模型之间的性能差距通常并不显著，导致将知识从一个模型蒸馏至另一个模型的收益有限。基于这些考量，互学习方法可能更具优势，因为所有模型可通过相互学习实现共同提升。本文通过实验证明，在对等模型间应用互学习算法具有显著价值。在Criteo和Avazu数据集上的实验表明，互学习算法使模型性能最高获得0.66%的相对提升。