This paper proposes a general interpretable predictive system with shared information. The system is able to perform predictions in a multi-task setting where distinct tasks are not bound to have the same input/output structure. Embeddings of input and output variables in a common space are obtained, where the input embeddings are produced through attending to a set of shared embeddings, reused across tasks. All the embeddings are treated as model parameters and learned. Specific restrictions on the space of shared embedings and the sparsity of the attention mechanism are considered. Experiments show that the introduction of shared embeddings does not deteriorate the results obtained from a vanilla variable embeddings method. We run a number of further ablations. Inducing sparsity in the attention mechanism leads to both an increase in accuracy and a significant decrease in the number of training steps required. Shared embeddings provide a measure of interpretability in terms of both a qualitative assessment and the ability to map specific shared embeddings to pre-defined concepts that are not tailored to the considered model. There seems to be a trade-off between accuracy and interpretability. The basic shared embeddings method favors interpretability, whereas the sparse attention method promotes accuracy. The results lead to the conclusion that variable embedding methods may be extended with shared information to provide increased interpretability and accuracy.

翻译：本文提出了一种具有共享信息的通用可解释预测系统。该系统能够在多任务设置下执行预测，其中不同任务不要求具有相同的输入/输出结构。通过获取输入和输出变量在公共空间中的嵌入表示，其中输入嵌入是通过关注一组跨任务重用的共享嵌入而产生的。所有嵌入均被视为模型参数进行学习。研究考虑了共享嵌入空间的特定约束以及注意力机制的稀疏性。实验表明，引入共享嵌入不会降低基础变量嵌入方法所获得的结果。我们进行了多项进一步的消融实验。在注意力机制中引入稀疏性既能提高准确率，又能显著减少所需的训练步数。共享嵌入通过定性评估和将特定共享嵌入映射到非模型定制预定义概念的能力，提供了可解释性度量。准确率与可解释性之间似乎存在权衡关系：基础共享嵌入方法偏向可解释性，而稀疏注意力方法则提升准确率。结果表明，变量嵌入方法可通过扩展共享信息来提供更强的可解释性与准确率。