Deep convolutional neural networks are a powerful model class for a range of computer vision problems, but it is difficult to interpret the image filtering process they implement, given their sheer size. In this work, we introduce a method for extracting 'feature-preserving circuits' from deep CNNs, leveraging methods from saliency-based neural network pruning. These circuits are modular sub-functions, embedded within the network, containing only a subset of convolutional kernels relevant to a target feature. We compare the efficacy of 3 saliency-criteria for extracting these sparse circuits. Further, we show how 'sub-feature' circuits can be extracted, that preserve a feature's responses to particular images, dividing the feature into even sparser filtering processes. We also develop a tool for visualizing 'circuit diagrams', which render the entire image filtering process implemented by circuits in a parsable format.

翻译：深度卷积神经网络是解决一系列计算机视觉问题的强大模型类别，但由于其庞大的规模，很难解释其所实现的图像滤波过程。在本文中，我们引入了一种从深度CNN中提取"特征保持电路"的方法，该方法利用了基于显著性的神经网络剪枝技术。这些电路是嵌入在网络中的模块化子功能，仅包含与目标特征相关的部分卷积核。我们比较了三种用于提取这些稀疏电路的显著性准则的有效性。此外，我们展示了如何提取"子特征"电路，这些电路能保留特征对特定图像的响应，将特征分解为更稀疏的滤波过程。我们还开发了一种可视化"电路图"的工具，该工具以可解析的格式呈现电路所实现的整个图像滤波过程。