Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6 TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, research in this domain has focused on Self-Supervised Learning and Foundation Model approaches. This paper addresses the need to evaluate different Foundation Models on a fair and uniform benchmark by introducing the PhilEO Bench, a novel evaluation framework for EO Foundation Models. The framework comprises of a testbed and a novel 400 GB Sentinel-2 dataset containing labels for three downstream tasks, building density estimation, road segmentation, and land cover classification. We present experiments using our framework evaluating different Foundation Models, including Prithvi and SatMAE, at multiple n-shots and convergence rates.

翻译：地球观测（EO）卫星捕获了大量未标注数据，其中哨兵-2号星座每天生成1.6 TB数据。这使得遥感领域成为数据丰富、非常适合机器学习（ML）解决方案的领域。然而，将ML模型应用于EO的一个瓶颈是缺乏标注数据，因为标注过程既劳动密集又成本高昂。因此，该领域的研究主要集中在自监督学习和基础模型方法上。本文通过引入PhilEO Bench（一个用于EO基础模型的新型评估框架），解决了在公平统一的基准上评估不同基础模型的需求。该框架包含一个测试平台和一个新颖的400 GB哨兵-2号数据集，其中包含三项下游任务的标签：建筑密度估计、道路分割和土地覆盖分类。我们利用该框架进行了实验，评估了包括Prithvi和SatMAE在内的多种基础模型，并考察了多样本设置下的收敛速率。