In some medical imaging tasks and other settings where only small parts of the image are informative for the classification task, traditional CNNs can sometimes struggle to generalise. Manually annotated Regions of Interest (ROI) are sometimes used to isolate the most informative parts of the image. However, these are expensive to collect and may vary significantly across annotators. To overcome these issues, we propose a framework that employs saliency maps to obtain soft spatial attention masks that modulate the image features at different scales. We refer to our method as Adversarial Counterfactual Attention (ACAT). ACAT increases the baseline classification accuracy of lesions in brain CT scans from 71.39% to 72.55% and of COVID-19 related findings in lung CT scans from 67.71% to 70.84% and exceeds the performance of competing methods. We investigate the best way to generate the saliency maps employed in our architecture and propose a way to obtain them from adversarially generated counterfactual images. They are able to isolate the area of interest in brain and lung CT scans without using any manual annotations. In the task of localising the lesion location out of 6 possible regions, they obtain a score of 65.05% on brain CT scans, improving the score of 61.29% obtained with the best competing method.

翻译：在某些医学影像任务及其他仅图像中微小区域包含分类信息的场景中，传统CNN有时难以实现良好泛化。人工标注的感兴趣区域（ROI）常被用于隔离图像中最具信息量的部分，但这类标注成本高昂且标注者间差异显著。为克服这些问题，我们提出一种利用显著性图获取软空间注意力掩膜的框架，该掩膜可在不同尺度上调节图像特征。我们将该方法称为对抗性反事实注意力（ACAT）。ACAT将脑部CT扫描中病灶分类的基线准确率从71.39%提升至72.55%，肺部CT扫描中COVID-19相关发现的分类准确率从67.71%提升至70.84%，并超越了竞争方法的性能。我们研究了生成架构中采用显著性图的最佳方案，并提出从对抗性生成的"反事实图像"中获取显著性图的方法。该方法无需人工标注即可隔离脑部与肺部CT扫描中的感兴趣区域。在6个可能区域中定位病灶位置的任务中，该方法在脑部CT扫描上取得了65.05%的得分，优于最佳竞争方法61.29%的得分。