Energy-based learning is a powerful learning paradigm that encapsulates various discriminative and generative approaches. An energy-based model (EBM) is typically formed of inner-model(s) that learn a combination of the different features to generate an energy mapping for each input configuration. In this paper, we focus on the diversity of the produced feature set. We extend the probably approximately correct (PAC) theory of EBMs and analyze the effect of redundancy reduction on the performance of EBMs. We derive generalization bounds for various learning contexts, i.e., regression, classification, and implicit regression, with different energy functions and we show that indeed reducing redundancy of the feature set can consistently decrease the gap between the true and empirical expectation of the energy and boosts the performance of the model.

翻译：基于能量的学习是一种强大的学习范式，它包含了各种判别式和生成式方法。基于能量的模型（EBM）通常由内部模型组成，这些内部模型学习不同特征的组合，从而为每个输入配置生成能量映射。本文重点研究生成特征集的多样性。我们扩展了EBM的概率近似正确（PAC）理论，并分析了冗余减少对EBM性能的影响。我们针对不同学习场景（即回归、分类和隐式回归），推导了使用不同能量函数时的泛化界，并证明减少特征集的冗余确实能够持续缩小能量真实期望与经验期望之间的差距，从而提升模型性能。