We propose a deep neural network (DNN) based least distance (LD) estimator (DNN-LD) for a multivariate regression problem, addressing the limitations of the conventional methods. Due to the flexibility of a DNN structure, both linear and nonlinear conditional mean functions can be easily modeled, and a multivariate regression model can be realized by simply adding extra nodes at the output layer. The proposed method is more efficient in capturing the dependency structure among responses than the least squares loss, and robust to outliers. In addition, we consider $L_1$-type penalization for variable selection, crucial in analyzing high-dimensional data. Namely, we propose what we call (A)GDNN-LD estimator that enjoys variable selection and model estimation simultaneously, by applying the (adaptive) group Lasso penalty to weight parameters in the DNN structure. For the computation, we propose a quadratic smoothing approximation method to facilitate optimizing the non-smooth objective function based on the least distance loss. The simulation studies and a real data analysis demonstrate the promising performance of the proposed method.

翻译：我们提出一种基于深度神经网络（DNN）的最小距离（LD）估计器（DNN-LD），用于解决多变量回归问题，弥补了传统方法的局限性。由于DNN结构的灵活性，线性与非线性条件均值函数均可被轻松建模，且仅需在输出层增加额外节点即可实现多变量回归模型。相较于最小二乘损失，所提方法在捕捉响应变量间的依赖结构方面更为高效，且对异常值具有鲁棒性。此外，我们考虑采用$L_1$型惩罚项进行变量选择，这对高维数据分析至关重要。具体而言，我们提出所谓（自适应）组DNN-LD估计器（(A)GDNN-LD），通过对DNN结构中的权重参数施加（自适应）组Lasso惩罚，同时实现变量选择与模型估计。在计算方面，我们提出一种二次平滑近似方法，以优化基于最小距离损失的非光滑目标函数。仿真实验与真实数据分析表明，所提方法具有卓越性能。