Machine learning enables extracting clinical insights from large temporal datasets. The applications of such machine learning models include identifying disease patterns and predicting patient outcomes. However, limited interpretability poses challenges for deploying advanced machine learning in digital healthcare. Understanding the meaning of latent states is crucial for interpreting machine learning models, assuming they capture underlying patterns. In this paper, we present a concise algorithm that allows for i) interpreting latent states using highly related input features; ii) interpreting predictions using subsets of input features via latent states; and iii) interpreting changes in latent states over time. The proposed algorithm is feasible for any model that is differentiable. We demonstrate that this approach enables the identification of a daytime behavioral pattern for predicting nocturnal behavior in a real-world healthcare dataset.

翻译：机器学习能从大规模时间序列数据中提取临床见解，这类模型的应用包括识别疾病模式和预测患者结局。然而，有限的解释性给高级机器学习在数字医疗中的部署带来了挑战。理解潜在状态的含义对于解释机器学习模型至关重要——假设它们捕捉到了底层模式。本文提出了一种简洁算法，可实现：i) 通过高度相关的输入特征解释潜在状态；ii) 通过潜在状态利用输入特征子集解释预测结果；iii) 解释潜在状态随时间的变化。该算法适用于任何可微模型。我们通过真实医疗数据集证明，该方法能够识别日间行为模式以预测夜间行为。