Carefully curated and annotated datasets are the foundation of machine learning, with particularly data-hungry deep neural networks forming the core of what is often called Artificial Intelligence (AI). Due to the massive success of deep learning applied to Earth Observation (EO) problems, the focus of the community has been largely on the development of ever-more sophisticated deep neural network architectures and training strategies largely ignoring the overall importance of datasets. For that purpose, numerous task-specific datasets have been created that were largely ignored by previously published review articles on AI for Earth observation. With this article, we want to change the perspective and put machine learning datasets dedicated to Earth observation data and applications into the spotlight. Based on a review of the historical developments, currently available resources are described and a perspective for future developments is formed. We hope to contribute to an understanding that the nature of our data is what distinguishes the Earth observation community from many other communities that apply deep learning techniques to image data, and that a detailed understanding of EO data peculiarities is among the core competencies of our discipline.

翻译：精心整理和标注的数据集是机器学习的基石，而尤其依赖数据的深度神经网络更是所谓人工智能（AI）的核心。由于深度学习在地球观测问题上的巨大成功，该领域的研究重点主要集中于开发日益复杂的深度神经网络架构和训练策略，而普遍忽视了数据集的整体重要性。为此，已创建了大量面向特定任务的数据集，但这些数据集在很大程度上被先前发表的关于地球观测人工智能的综述文章所忽略。通过本文，我们希望转变视角，将专门用于地球观测数据和应用的数据集置于聚光灯下。在回顾历史发展的基础上，我们描述了当前可用的资源，并形成了对未来发展的展望。我们希望能促进这样一种认识：数据的本质是将地球观测领域与许多其他应用深度学习技术处理图像数据的领域区分开来的关键，而对地球观测数据特性的深入理解正是我们学科的核心能力之一。