Sparse linear models are one of several core tools for interpretable machine learning, a field of emerging importance as predictive models permeate decision-making in many domains. Unfortunately, sparse linear models are far less flexible as functions of their input features than black-box models like deep neural networks. With this capability gap in mind, we study a not-uncommon situation where the input features dichotomize into two groups: explanatory features, which are candidates for inclusion as variables in an interpretable model, and contextual features, which select from the candidate variables and determine their effects. This dichotomy leads us to the contextual lasso, a new statistical estimator that fits a sparse linear model to the explanatory features such that the sparsity pattern and coefficients vary as a function of the contextual features. The fitting process learns this function nonparametrically via a deep neural network. To attain sparse coefficients, we train the network with a novel lasso regularizer in the form of a projection layer that maps the network's output onto the space of $\ell_1$-constrained linear models. An extensive suite of experiments on real and synthetic data suggests that the learned models, which remain highly transparent, can be sparser than the regular lasso without sacrificing the predictive power of a standard deep neural network.

翻译：稀疏线性模型是可解释机器学习的关键工具之一，随着预测模型渗透到多个领域的决策过程中，这一领域的重要性日益凸显。然而，稀疏线性模型作为输入特征的函数，其灵活性远不及深度神经网络等黑箱模型。基于这一能力差距，我们研究了一个并不罕见的情形：输入特征分为两组——解释性特征（作为可解释模型中候选变量的特征）和上下文特征（用于选择候选变量并决定其效应）。这种二分法引出了上下文套索（contextual lasso），一种新的统计估计量，它对解释性特征拟合稀疏线性模型，使稀疏模式和系数随上下文特征的变化而变化。拟合过程通过深度神经网络以非参数方式学习这一函数。为获得稀疏系数，我们使用一种新型套索正则化器训练网络，该正则化器以投影层的形式将网络输出映射到ℓ₁约束线性模型空间。在真实和合成数据上进行的一系列广泛实验表明，学习到的模型保持高度透明性，其稀疏度可通过常规套索，且不牺牲标准深度神经网络的预测能力。