Massive amounts of unlabelled data are captured by Earth Observation (EO) satellites, with the Sentinel-2 constellation generating 1.6 TB of data daily. This makes Remote Sensing a data-rich domain well suited to Machine Learning (ML) solutions. However, a bottleneck in applying ML models to EO is the lack of annotated data as annotation is a labour-intensive and costly process. As a result, research in this domain has focused on Self-Supervised Learning and Foundation Model approaches. This paper addresses the need to evaluate different Foundation Models on a fair and uniform benchmark by introducing the PhilEO Bench, a novel evaluation framework for EO Foundation Models. The framework comprises of a testbed and a novel 400 GB Sentinel-2 dataset containing labels for three downstream tasks, building density estimation, road segmentation, and land cover classification. We present experiments using our framework evaluating different Foundation Models, including Prithvi and SatMAE, at multiple n-shots and convergence rates.

翻译：地球观测卫星捕捉到海量的未标注数据，其中哨兵2号星座每天生成1.6 TB数据。这使得遥感成为数据丰富且适合机器学习解决方案的领域。然而，将机器学习模型应用于地球观测的一个瓶颈是缺乏标注数据，因为标注是一个劳动密集且成本高昂的过程。因此，该领域的研究聚焦于自监督学习和基础模型方法。本文通过引入PhilEO基准（一种新颖的地球观测基础模型评估框架），解决了在不同基础模型之间进行公平统一评估的需求。该框架包含一个测试平台和一个全新的400 GB哨兵2号数据集，该数据集包含三个下游任务的标注：建筑密度估计、道路分割和土地覆盖分类。我们利用该框架进行了实验，评估了包括Prithvi和SatMAE在内的多种基础模型，并分析了不同样本量和收敛率下的性能表现。