Accurate and timely crop mapping is essential for yield estimation, insurance claims, and conservation efforts. Over the years, many successful machine learning models for crop mapping have been developed that use just the multi-spectral imagery from satellites to predict crop type over the area of interest. However, these traditional methods do not account for the physical processes that govern crop growth. At a high level, crop growth can be envisioned as physical parameters, such as weather and soil type, acting upon the plant leading to crop growth which can be observed via satellites. In this paper, we propose Weather-based Spatio-Temporal segmentation network with ATTention (WSTATT), a deep learning model that leverages this understanding of crop growth by formulating it as an inverse model that combines weather (Daymet) and satellite imagery (Sentinel-2) to generate accurate crop maps. We show that our approach provides significant improvements over existing algorithms that solely rely on spectral imagery by comparing segmentation maps and F1 classification scores. Furthermore, effective use of attention in WSTATT architecture enables detection of crop types earlier in the season (up to 5 months in advance), which is very useful for improving food supply projections. We finally discuss the impact of weather by correlating our results with crop phenology to show that WSTATT is able to capture physical properties of crop growth.

翻译：精准及时的作物制图对于产量估算、保险理赔及资源保护至关重要。近年来，许多成功的作物制图机器学习模型仅利用卫星多光谱影像即可预测感兴趣区域的作物类型。然而，这些传统方法未能考虑控制作物生长的物理过程。从宏观层面看，作物生长可视为天气、土壤类型等物理参数作用于植株，最终通过卫星观测到的生长结果。本文提出基于天气的时空注意力分割网络（WSTATT）——一种深度学习模型，通过将作物生长过程建模为逆模型，融合天气数据（Daymet）与卫星影像（Sentinel-2）生成精准作物地图。通过对比分割图与F1分类分数，我们证明该方法相比仅依赖光谱影像的现有算法具有显著优势。此外，WSTATT架构中注意力的有效运用使得作物类型可在生长季早期（提前多达5个月）被识别，这对改进粮食供应预测极具价值。最后，通过将结果与作物物候期相关联分析天气影响，证明WSTATT能够捕捉作物生长的物理属性。