Difficulties may arise when analyzing longitudinal data using mixed-effects models if there are nonparametric functions present in the linear predictor component. This study extends the use of semiparametric mixed-effects modeling in cases when the response variable does not always follow a normal distribution and the nonparametric component is structured as an additive model. A novel approach is proposed to identify significant linear and non-linear components using a double-penalized generalized estimating equation with two penalty terms. Furthermore, the iterative approach provided intends to enhance the efficiency of estimating regression coefficients by incorporating the calculation of the working covariance matrix. The oracle properties of the resulting estimators are established under certain regularity conditions, where the dimensions of both the parametric and nonparametric components increase as the sample size grows. We perform numerical studies to demonstrate the efficacy of our proposal.

翻译：纵向数据分析中，若线性预测部分存在非参数函数，采用混合效应模型时可能面临困难。本研究将半参数混合效应模型的应用扩展到响应变量不服从正态分布且非参数部分以加性模型形式存在的情形。针对含双重惩罚项的广义估计方程，我们提出一种新颖方法以识别显著的线性与非线性分量。此外，通过引入工作协方差矩阵的计算，所提出的迭代方法旨在提升回归系数的估计效率。在一定正则条件下，当参数与非参数分量维度随样本量增长而增加时，所得估计量具有oracle性质。数值研究验证了所提方法的有效性。