Sparse linear models are a gold standard tool for interpretable machine learning, a field of emerging importance as predictive models permeate decision-making in many domains. Unfortunately, sparse linear models are far less flexible as functions of their input features than black-box models like deep neural networks. With this capability gap in mind, we study a not-uncommon situation where the input features dichotomize into two groups: explanatory features, which we wish to explain the model's predictions, and contextual features, which we wish to determine the model's explanations. This dichotomy leads us to propose the contextual lasso, a new statistical estimator that fits a sparse linear model whose sparsity pattern and coefficients can vary with the contextual features. The fitting process involves learning a nonparametric map, realized via a deep neural network, from contextual feature vector to sparse coefficient vector. To attain sparse coefficients, we train the network with a novel lasso regularizer in the form of a projection layer that maps the network's output onto the space of $\ell_1$-constrained linear models. Extensive experiments on real and synthetic data suggest that the learned models, which remain highly transparent, can be sparser than the regular lasso without sacrificing the predictive power of a standard deep neural network.

翻译：稀疏线性模型是可解释机器学习中的黄金标准工具，随着预测模型渗透到许多领域的决策中，这一领域的重要性日益凸显。然而，与深度神经网络等黑箱模型相比，稀疏线性模型作为输入特征函数的灵活性远不及前者。基于这一能力差距，我们研究了一种并不罕见的情形：输入特征可分为两组——解释性特征（用于解释模型预测）和上下文特征（用于确定模型解释）。这一二分法促使我们提出“上下文Lasso”，这是一种新的统计估计量，它拟合一个稀疏线性模型，其稀疏模式和系数可随上下文特征变化。拟合过程涉及学习一个由深度神经网络实现的非参数映射，从上下文特征向量映射到稀疏系数向量。为了获得稀疏系数，我们使用一种新颖的Lasso正则化器训练网络，该正则化器以投影层的形式实现，将网络输出映射到$\ell_1$约束线性模型的空间中。在真实和合成数据上的大量实验表明，所学模型保持高度透明，可较普通Lasso更稀疏，同时不牺牲标准深度神经网络的预测能力。