The influence of natural image transformations on receptive field responses is crucial for modelling visual operations in computer vision and biological vision. In this regard, covariance properties with respect to geometric image transformations in the earliest layers of the visual hierarchy are essential for expressing robust image operations, and for formulating invariant visual operations at higher levels. This paper defines and proves a set of joint covariance properties under compositions of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations, which make it possible to characterize how different types of image transformations interact with each other and the associated spatio-temporal receptive field responses. In this regard, we also extend the notion of scale-normalized derivatives to affine-normalized derivatives, to be able to obtain true affine-covariant properties of spatial derivatives, that are computed based on spatial smoothing with affine Gaussian kernels. The derived relations show how the parameters of the receptive fields need to be transformed, in order to match the output from spatio-temporal receptive fields under composed spatio-temporal image transformations. As a side effect, the presented proof for the joint covariance property over the integrated combination of the different geometric image transformations also provides specific proofs for the individual transformation properties, which have not previously been fully reported in the literature. The paper also presents an in-depth theoretical analysis of geometric interpretations of the derived covariance properties, as well as outlines a number of biological interpretations of these results.
翻译:自然图像变换对感受野响应的影响对于计算机视觉和生物视觉中的视觉操作建模至关重要。在此方面,视觉层级最早期层中关于几何图像变换的协方差性质对于表达鲁棒的图像操作以及在高层次上构建不变性视觉操作具有关键作用。本文定义并证明了在空间尺度变换、空间仿射变换、伽利略变换和时间尺度变换组合下的一组联合协方差性质,这些性质能够刻画不同类型图像变换如何相互作用以及与之相关的时空感受野响应。为此,我们还将尺度归一化导数的概念扩展至仿射归一化导数,从而获得基于仿射高斯核空间平滑计算的空间导数的真实仿射协方差性质。推导的关系展示了在组合时空图像变换下,为匹配时空感受野输出,感受野参数需要如何变换。作为附带结果,本文针对不同几何图像变换整合组合的联合协方差性质证明,也同时提供了这些单独变换性质的具体证明——这些证明此前在文献中尚未完整报告。此外,本文对推导的协方差性质的几何解释进行了深入的理论分析,并概述了这些结果的多种生物学解释。