Visual counterfactual explanations (VCEs) have recently gained immense popularity as a tool for clarifying the decision-making process of image classifiers. This trend is largely motivated by what these explanations promise to deliver -- indicate semantically meaningful factors that change the classifier's decision. However, we argue that current state-of-the-art approaches lack a crucial component -- the region constraint -- whose absence prevents from drawing explicit conclusions, and may even lead to faulty reasoning due to phenomenons like confirmation bias. To address the issue of previous methods, which modify images in a very entangled and widely dispersed manner, we propose region-constrained VCEs (RVCEs), which assume that only a predefined image region can be modified to influence the model's prediction. To effectively sample from this subclass of VCEs, we propose Region-Constrained Counterfactual Schr\"odinger Bridges (RCSB), an adaptation of a tractable subclass of Schr\"odinger Bridges to the problem of conditional inpainting, where the conditioning signal originates from the classifier of interest. In addition to setting a new state-of-the-art by a large margin, we extend RCSB to allow for exact counterfactual reasoning, where the predefined region contains only the factor of interest, and incorporating the user to actively interact with the RVCE by predefining the regions manually.

翻译：视觉反事实解释（VCEs）作为一种阐明图像分类器决策过程的工具，近年来获得了极大的关注。这一趋势主要源于此类解释所承诺实现的目标——揭示能够改变分类器决策的语义层面关键因素。然而，我们认为当前最先进的方法缺乏一个关键组件——区域约束——其缺失使得我们无法得出明确的结论，甚至可能因确认偏误等现象导致错误推理。针对先前方法以高度纠缠且广泛分散的方式修改图像的问题，我们提出区域约束型视觉反事实解释（RVCEs），其假设仅允许修改预定义的图像区域以影响模型预测。为有效采样此类VCEs的子类，我们提出区域约束反事实薛定谔桥（RCSB），该方法将薛定谔桥的一个可处理子类适配于条件修复问题，其中条件信号源自目标分类器。除了以显著优势确立新的技术标杆外，我们还扩展了RCSB以实现精确反事实推理（此时预定义区域仅包含目标因子），并通过支持用户手动预定义区域来实现与RVCEs的主动交互。