Sparse linear models are one of several core tools for interpretable machine learning, a field of emerging importance as predictive models permeate decision-making in many domains. Unfortunately, sparse linear models are far less flexible as functions of their input features than black-box models like deep neural networks. With this capability gap in mind, we study a not-uncommon situation where the input features dichotomize into two groups: explanatory features, which are candidates for inclusion as variables in an interpretable model, and contextual features, which select from the candidate variables and determine their effects. This dichotomy leads us to the contextual lasso, a new statistical estimator that fits a sparse linear model to the explanatory features such that the sparsity pattern and coefficients vary as a function of the contextual features. The fitting process learns this function nonparametrically via a deep neural network. To attain sparse coefficients, we train the network with a novel lasso regularizer in the form of a projection layer that maps the network's output onto the space of $\ell_1$-constrained linear models. An extensive suite of experiments on real and synthetic data suggests that the learned models, which remain highly transparent, can be sparser than the regular lasso without sacrificing the predictive power of a standard deep neural network.

翻译：稀疏线性模型是可解释机器学习中的核心工具之一，随着预测模型渗透到众多领域的决策过程中，这一领域正日益重要。然而，稀疏线性模型作为输入特征的函数，其灵活性远不如深度神经网络等黑箱模型。针对这一能力差距，我们研究了一种并不罕见的情形：输入特征分为两组——解释性特征（作为可解释模型中变量选择的候选）和上下文特征（用于选择候选变量并决定其影响）。这种二分法引出了上下文Lasso，这是一种新的统计估计量，它能够对解释性特征拟合一个稀疏线性模型，使得稀疏性模式和系数随上下文特征的变化而改变。拟合过程通过深度神经网络以非参数方式学习这一函数。为了获得稀疏系数，我们使用一种新型的Lasso正则化器来训练网络，该正则化器采用投影层形式，将网络输出映射到$\ell_1$约束线性模型的空间中。在真实数据和合成数据上进行的大量实验表明，所学模型在保持高度透明性的同时，其稀疏性可能优于常规Lasso，且不牺牲标准深度神经网络的预测能力。