Drawing parallels with the way biological networks are studied, the authors adapt the treatment--control paradigm to explainable artificial intelligence research and enrich it through multi-parametric input alterations. A framework for investigating internal inference impacted by input data augmentations is proposed to address the real-world scenario with multiple simultaneously applied treatments instead of observing a single effect. The internal changes in network operation are reflected in activation changes measured by variance, which can be decomposed into components related to each augmentation by employing Sobol indices and Shapley values. To facilitate distinguishing model-related specifics from the self-interactions of augmentations, an auxiliary variable responsible for their permutations has been incorporated into the analysis. The resulting quantities enable one to visualize sensitivity to different variables and use them as features for further analysis and processing, including guided masking of activations. In addition, a method for single-class sensitivity analysis is introduced through a pipeline that involves class selection according to their match with the prediction bias generated by targeted damaging of the activations. Three convolutional networks, namely AlexNet, VGG11, and ResNet18, have been evaluated via the designed framework, with two vision transformers, ViT-b-16 and Swin-t, partially complementing the findings. Two sets of augmentations, consisting of five and seven distinct input transformations applied simultaneously, have been utilized as sources of variance in the image domain. Relying on the observed parallels, the authors assume that the developed framework can potentially be transferred to the study of biological neural networks in complex environments, as well as enhance the design of autonomous decision-making systems.

翻译：暂无翻译