As deep learning predictive models become an integral part of a large spectrum of precision agricultural systems, a barrier to the adoption of such automated solutions is the lack of user trust in these highly complex, opaque and uncertain models. Indeed, deep neural networks are not equipped with any explicit guarantees that can be used to certify the system's performance, especially in highly varying uncontrolled environments such as the ones typically faced in computer vision for agriculture.Fortunately, certain methods developed in other communities can prove to be important for agricultural applications. This article presents the conformal prediction framework that provides valid statistical guarantees on the predictive performance of any black box prediction machine, with almost no assumptions, applied to the problem of deep visual classification of weeds and crops in real-world conditions. The framework is exposed with a focus on its practical aspects and special attention accorded to the Adaptive Prediction Sets (APS) approach that delivers marginal guarantees on the model's coverage. Marginal results are then shown to be insufficient to guarantee performance on all groups of individuals in the population as characterized by their environmental and pedo-climatic auxiliary data gathered during image acquisition.To tackle this shortcoming, group-conditional conformal approaches are presented: the ''classical'' method that consists of iteratively applying the APS procedure on all groups, and a proposed elegant reformulation and implementation of the procedure using quantile regression on group membership indicators. Empirical results showing the validity of the proposed approach are presented and compared to the marginal APS then discussed.
翻译:随着深度学习预测模型成为精准农业系统的重要组成部分,这类自动化解决方案的推广应用面临一个障碍:用户对这些高度复杂、不透明且不确定的模型缺乏信任。事实上,深度神经网络并不具备任何明确保证来认证系统性能,尤其在计算机视觉农业应用中常见的、高度多变且不受控的环境下。幸运的是,其他领域开发的某些方法对农业应用具有重要价值。本文介绍了共形预测框架,该框架能在几乎无假设条件下,为任意黑箱预测模型的预测性能提供有效的统计保证,并将其应用于真实条件下杂草与作物的深度视觉分类问题。本文着重阐述该框架的实践层面,特别关注自适应预测集(APS)方法,该方法为模型覆盖度提供边际保证。研究表明,边际结果不足以确保对群体中所有个体组别的性能保障——这些组别由图像采集期间收集的环境和土壤气候辅助数据刻画。为克服这一缺陷,本文提出组条件共形方法:一是对所有组别迭代应用APS流程的"经典"方法,二是通过组隶属指标的分位数回归对该流程进行优雅的重新表述与实现。本文展示了所提方法的有效性实证结果,并与边际APS进行比较和讨论。