Generalization remains a central challenge in machine learning. In this work, we propose Learning from Teaching (LoT), a novel regularization technique for deep neural networks to enhance generalization. Inspired by the human ability to capture concise and abstract patterns, we hypothesize that generalizable correlations are expected to be easier to teach. LoT operationalizes this concept to improve the generalization of the main model with auxiliary student learners. The student learners are trained by the main model and improve the main model to capture more generalizable and teachable correlations by providing feedback. Our experimental results across several domains, including Computer Vision, Natural Language Processing, and Reinforcement Learning, demonstrate that the introduction of LoT brings significant benefits compared to merely training models on the original training data. It suggests the effectiveness of LoT in identifying generalizable information without falling into the swamp of complex patterns in data, making LoT a valuable addition to the current machine learning frameworks.

翻译：泛化能力仍是机器学习领域的核心挑战。本研究提出"从教学中学习"（Learning from Teaching, LoT）——一种面向深度神经网络的新型正则化技术，旨在增强模型泛化性能。受人类捕捉简洁抽象模式的能力启发，我们假设可泛化的相关性应当更易于被传授。LoT通过辅助学生模型实现这一理念，以提升主模型的泛化能力。学生模型由主模型训练而来，通过提供反馈促使主模型捕捉更具泛化性和可教性的相关性。我们在计算机视觉、自然语言处理与强化学习等多个领域的实验结果表明，相较于仅在原始训练数据上训练模型，引入LoT能带来显著性能提升。这验证了LoT在识别可泛化信息方面的有效性——既能避免陷入数据中复杂模式的迷沼，又使其成为当前机器学习框架中极具价值的技术补充。