Recent advances to combine structured regression models and deep neural networks for better interpretability, more expressiveness, and statistically valid uncertainty quantification demonstrate the versatility of semi-structured neural networks (SSNs). We show that techniques to properly identify the contributions of the different model components in SSNs, however, lead to suboptimal network estimation, slower convergence, and degenerated or erroneous predictions. In order to solve these problems while preserving favorable model properties, we propose a non-invasive post-hoc orthogonalization (PHO) that guarantees identifiability of model components and provides better estimation and prediction quality. Our theoretical findings are supported by numerical experiments, a benchmark comparison as well as a real-world application to COVID-19 infections.

翻译：近年来的进展将结构化回归模型与深度神经网络相结合，以实现更高的可解释性、更强的表达能力及具有统计有效性的不确定性量化，充分展示了半结构化神经网络（SSN）的通用性。然而，我们证明：当前用于正确识别SSN中不同模型组成部分贡献的技术，会导致网络估计次优、收敛速度减慢以及预测能力退化或产生错误预测。为解决上述问题并保留模型优良特性，我们提出一种非侵入式事后正交化方法（PHO），该方法既能保证模型组成部分的可识别性，又能提供更优的估计与预测质量。我们的理论发现通过数值实验、基准对比以及一项关于COVID-19感染的真实应用得到了验证。