Personalized prediction is a machine learning approach that predicts a person's future observations based on their past labeled observations and is typically used for sequential tasks, e.g., to predict daily mood ratings. When making personalized predictions, a model can combine two types of trends: (a) trends shared across people, i.e., person-generic trends, such as being happier on weekends, and (b) unique trends for each person, i.e., person-specific trends, such as a stressful weekly meeting. Mixed effect models are popular statistical models to study both trends by combining person-generic and person-specific parameters. Though linear mixed effect models are gaining popularity in machine learning by integrating them with neural networks, these integrations are currently limited to linear person-specific parameters: ruling out nonlinear person-specific trends. In this paper, we propose Neural Mixed Effect (NME) models to optimize nonlinear person-specific parameters anywhere in a neural network in a scalable manner. NME combines the efficiency of neural network optimization with nonlinear mixed effects modeling. Empirically, we observe that NME improves performance across six unimodal and multimodal datasets, including a smartphone dataset to predict daily mood and a mother-adolescent dataset to predict affective state sequences where half the mothers experience at least moderate symptoms of depression. Furthermore, we evaluate NME for two model architectures, including for neural conditional random fields (CRF) to predict affective state sequences where the CRF learns nonlinear person-specific temporal transitions between affective states. Analysis of these person-specific transitions on the mother-adolescent dataset shows interpretable trends related to the mother's depression symptoms.

翻译：个性化预测是一种机器学习方法，通过个体过去的标注观测数据预测其未来观测值，通常用于序列任务，例如预测每日情绪评分。在进行个性化预测时，模型需结合两类趋势：（a）人群共享趋势，即普适性趋势（如周末更快乐）；（b）个体专属趋势，即特异性趋势（如每周压力性会议）。混合效应模型通过融合普适性与特异性参数，成为研究这两类趋势的经典统计学方法。尽管线性混合效应模型通过神经网络集成在机器学习领域日益普及，但这种集成目前局限于线性个体特异性参数，排除了非线性个体特异性趋势。本文提出神经混合效应（NME）模型，以可扩展的方式在神经网络任意位置优化非线性个体特异性参数。NME结合了神经网络优化的高效性与非线性混合效应建模的优势。实验表明，NME在六个单模态与多模态数据集上均提升了性能，包括用于预测每日情绪的手机数据集和预测情感状态序列的母子数据集（其中半数母亲存在至少中度抑郁症状）。此外，我们评估了NME在两种模型架构中的表现，包括用于预测情感状态序列的神经条件随机场（CRF），其中CRF可学习情感状态间的非线性个体特异性时序转移。针对母子数据集中个体特异性转移模式的分析揭示了与母亲抑郁症状相关的可解释趋势。