Generalization remains a central challenge in machine learning. In this work, we propose Learning from Teaching (LoT), a novel regularization technique for deep neural networks to enhance generalization. Inspired by the human ability to capture concise and abstract patterns, we hypothesize that generalizable correlations are expected to be easier to imitate. LoT operationalizes this concept to improve generalization of the main model with auxiliary student learners. The student learners are trained by the main model and, in turn, provide feedback to help the main model capture more generalizable and imitable correlations. Our experimental results across several domains, including Computer Vision, Natural Language Processing, and methodologies like Reinforcement Learning, demonstrate that the introduction of LoT brings significant benefits compared to training models on the original dataset. The results suggest the effectiveness and efficiency of LoT in identifying generalizable information at the right scales while discarding spurious data correlations, thus making LoT a valuable addition to current machine learning. Code is available at https://github.com/jincan333/LoT.

翻译：泛化能力仍然是机器学习的核心挑战。本文提出了一种新颖的深度神经网络正则化技术——从教学中学习（LoT），以增强模型的泛化性能。受人类捕捉简洁抽象模式能力的启发，我们假设可泛化的相关性应当更易于被模仿。LoT将这一概念操作化，通过辅助学生学习器来提升主模型的泛化能力。学生学习器由主模型训练，并反过来提供反馈，帮助主模型捕获更具泛化性和可模仿性的相关性。我们在多个领域（包括计算机视觉、自然语言处理）和方法论（如强化学习）中的实验结果表明，与在原始数据集上训练模型相比，引入LoT能带来显著优势。这些结果证明了LoT在识别适当尺度下可泛化信息、同时摒弃虚假数据相关性方面的有效性和高效性，使得LoT成为当前机器学习方法的有价值补充。代码发布于 https://github.com/jincan333/LoT。