Recent advances in large-scale models, including deep neural networks and large language models, have substantially improved performance across a wide range of learning tasks. The widespread availability of such pre-trained models creates new opportunities for data-efficient statistical learning, provided they can be effectively integrated into downstream tasks. Motivated by this setting, we study few-shot personalization, where a pre-trained black-box model is adapted to a target domain using a limited number of samples. We develop a theoretical framework for few-shot personalization in nonparametric regression and propose algorithms that can incorporate a black-box pre-trained model into the regression procedure. We establish the minimax optimal rate for the personalization problem and show that the proposed method attains this rate. Our results clarify the statistical benefits of leveraging pre-trained models under sample scarcity and provide robustness guarantees when the pre-trained model is not informative. We illustrate the finite-sample performance of the methods through simulations and an application to the California housing dataset with several pre-trained models.

翻译：近年来，大规模模型（包括深度神经网络和大语言模型）的进展显著提升了各类学习任务的性能。此类预训练模型的广泛可获得性为数据高效的统计学习创造了新的机遇，前提是它们能够被有效地整合到下游任务中。受此背景启发，我们研究小样本个性化问题，即利用有限样本将预训练的黑盒模型适配到目标领域。我们为非参数回归中的小样本个性化建立了一个理论框架，并提出了能够将黑盒预训练模型整合到回归过程中的算法。我们确立了该个性化问题的极小极大最优收敛速率，并证明了所提方法能够达到该速率。我们的结果阐明了在样本稀缺条件下利用预训练模型的统计优势，并在预训练模型信息量不足时提供了鲁棒性保证。我们通过仿真实验以及在加利福尼亚住房数据集上结合多种预训练模型的应用，展示了所提方法的有限样本性能。