The statistical supervised learning framework assumes an input-output set with a joint probability distribution that is reliably represented by the training dataset. The learner is then required to output a prediction rule learned from the training dataset's input-output pairs. In this work, we provide meaningful insights into the asymptotic equipartition property (AEP) \citep{Shannon:1948} in the context of machine learning, and illuminate some of its potential ramifications for few-shot learning. We provide theoretical guarantees for reliable learning under the information-theoretic AEP, and for the generalization error with respect to the sample size. We then focus on a highly efficient recurrent neural net (RNN) framework and propose a reduced-entropy algorithm for few-shot learning. We also propose a mathematical intuition for the RNN as an approximation of a sparse coding solver. We verify the applicability, robustness, and computational efficiency of the proposed approach with image deblurring and optical coherence tomography (OCT) speckle suppression. Our experimental results demonstrate significant potential for improving learning models' sample efficiency, generalization, and time complexity, that can therefore be leveraged for practical real-time applications.

翻译：统计监督学习框架假设输入-输出集具有联合概率分布，该分布可由训练数据集可靠表征。学习器需要从训练数据集的输入-输出对中学习并输出预测规则。本文从机器学习角度深入阐释渐近均分性质（AEP）\citep{Shannon:1948}，并阐明其对少样本学习的潜在影响。我们基于信息论框架下的AEP为可靠学习提供了理论保障，并建立了样本规模与泛化误差之间的理论关联。进一步聚焦高效循环神经网络（RNN）架构，提出了一种用于少样本学习的降熵算法。同时，我们提出了将RNN作为稀疏编码求解器近似模型的数学直觉。通过图像去模糊和光学相干断层扫描（OCT）散斑抑制实验，验证了所提方法的适用性、鲁棒性和计算效率。实验结果表明，该方法在提升学习模型样本效率、泛化能力和时间复杂度方面具有显著潜力，可为实际实时应用提供支撑。