Neural networks are powerful function approximators, yet their ``black-box" nature often renders them opaque and difficult to interpret. While many post-hoc explanation methods exist, they typically fail to capture the underlying reasoning processes of the networks. A truly interpretable neural network would be trained similarly to conventional models using techniques such as backpropagation, but additionally provide insights into the learned input-output relationships. In this work, we introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. To this end, we first evaluate several architectures that promise such interpretability, with a particular focus on two recent models selected for their potential to incorporate interpretability into standard neural network architectures while still leveraging backpropagation: the Growing Interpretable Neural Network (GINN) and Kolmogorov Arnold Networks (KAN). We analyze the limitations and strengths of each and introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models. When tested on the Feynman symbolic regression benchmark datasets, GINN-KAN outperforms both GINN and KAN. To highlight the capabilities and the generalizability of this approach, we position GINN-KAN as an alternative to conventional black-box networks in Physics-Informed Neural Networks (PINNs). We expect this to have far-reaching implications in the application of deep learning pipelines in the natural sciences. Our experiments with this interpretable PINN on 15 different partial differential equations demonstrate that GINN-KAN augmented PINNs outperform PINNs with black-box networks in solving differential equations and surpass the capabilities of both GINN and KAN.

翻译：神经网络是强大的函数逼近器，但其“黑盒”特性往往使其不透明且难以解释。尽管存在许多事后解释方法，它们通常无法捕捉网络底层的推理过程。一个真正可解释的神经网络应能像传统模型一样通过反向传播等技术进行训练，同时提供对所学输入输出关系的深入洞察。本文中，我们引入可解释性流水线的概念，通过整合多种可解释性技术以超越单一技术的性能。为此，我们首先评估了若干具备此类可解释性潜力的架构，特别聚焦于两种近期提出的模型——因其能在保持反向传播机制的同时将可解释性融入标准神经网络架构而被选中：增长可解释神经网络（GINN）与柯尔莫哥洛夫-阿诺德网络（KAN）。我们分析了各自的局限与优势，并提出一种新型可解释神经网络GINN-KAN，它融合了两种模型的优点。在费曼符号回归基准数据集上的测试表明，GINN-KAN的性能优于GINN和KAN。为突显该方法的能力与泛化性，我们将GINN-KAN定位为物理信息神经网络（PINNs）中传统黑盒网络的替代方案。这预计将对深度学习流水线在自然科学中的应用产生深远影响。我们在15种不同偏微分方程上使用这种可解释PINN进行的实验表明，采用GINN-KAN增强的PINN在求解微分方程方面优于使用黑盒网络的PINN，并超越了GINN与KAN各自的能力。