Unpacking and comprehending how black-box machine learning algorithms make decisions has been a persistent challenge for researchers and end-users. Explaining time-series predictive models is useful for clinical applications with high stakes to understand the behavior of prediction models. However, existing approaches to explain such models are frequently unique to data where the features do not have a time-varying component. In this paper, we introduce WindowSHAP, a model-agnostic framework for explaining time-series classifiers using Shapley values. We intend for WindowSHAP to mitigate the computational complexity of calculating Shapley values for long time-series data as well as improve the quality of explanations. WindowSHAP is based on partitioning a sequence into time windows. Under this framework, we present three distinct algorithms of Stationary, Sliding and Dynamic WindowSHAP, each evaluated against baseline approaches, KernelSHAP and TimeSHAP, using perturbation and sequence analyses metrics. We applied our framework to clinical time-series data from both a specialized clinical domain (Traumatic Brain Injury - TBI) as well as a broad clinical domain (critical care medicine). The experimental results demonstrate that, based on the two quantitative metrics, our framework is superior at explaining clinical time-series classifiers, while also reducing the complexity of computations. We show that for time-series data with 120 time steps (hours), merging 10 adjacent time points can reduce the CPU time of WindowSHAP by 80% compared to KernelSHAP. We also show that our Dynamic WindowSHAP algorithm focuses more on the most important time steps and provides more understandable explanations. As a result, WindowSHAP not only accelerates the calculation of Shapley values for time-series data, but also delivers more understandable explanations with higher quality.

翻译：解析和理解黑盒机器学习算法的决策过程一直是研究者和终端用户面临的持续性挑战。解释时间序列预测模型对于高风险临床应用中理解预测模型行为具有重要意义。然而，现有解释方法通常仅适用于特征不具备时变特性的数据场景。本文提出WindowSHAP——基于Shapley值的时间序列分类器模型无关解释框架。我们旨在通过WindowSHAP降低长序列时间数据Shapley值计算的复杂度，并提升解释质量。该框架通过将序列划分为时间窗口实现，并在此基础上提出三种不同算法：平稳窗口SHAP、滑动窗口SHAP与动态窗口SHAP。我们采用扰动分析与序列分析指标，分别将各算法与基线方法KernelSHAP和TimeSHAP进行对比评估。通过应用于专科临床领域（创伤性脑损伤）及广域临床领域（重症监护医学）的临床时间序列数据，实验结果表明：基于两种量化指标，本框架在解释临床时间序列分类器方面表现更优，同时降低了计算复杂度。研究显示，对于120个时间步长的序列数据，合并10个相邻时间点可使WindowSHAP的CPU运行时间较KernelSHAP降低80%。此外，动态窗口SHAP算法能更聚焦于关键时间步长，提供更具可理解性的解释。因此，WindowSHAP不仅加速了时间序列数据Shapley值的计算过程，还能输出更易理解的高质量解释结果。