Personalized prediction is a machine learning approach that predicts a person's future observations based on their past labeled observations and is typically used for sequential tasks, e.g., to predict daily mood ratings. When making personalized predictions, a model can combine two types of trends: (a) trends shared across people, i.e., person-generic trends, such as being happier on weekends, and (b) unique trends for each person, i.e., person-specific trends, such as a stressful weekly meeting. Mixed effect models are popular statistical models to study both trends by combining person-generic and person-specific parameters. Though linear mixed effect models are gaining popularity in machine learning by integrating them with neural networks, these integrations are currently limited to linear person-specific parameters: ruling out nonlinear person-specific trends. In this paper, we propose Neural Mixed Effect (NME) models to optimize nonlinear person-specific parameters anywhere in a neural network in a scalable manner. NME combines the efficiency of neural network optimization with nonlinear mixed effects modeling. Empirically, we observe that NME improves performance across six unimodal and multimodal datasets, including a smartphone dataset to predict daily mood and a mother-adolescent dataset to predict affective state sequences where half the mothers experience at least moderate symptoms of depression. Furthermore, we evaluate NME for two model architectures, including for neural conditional random fields (CRF) to predict affective state sequences where the CRF learns nonlinear person-specific temporal transitions between affective states. Analysis of these person-specific transitions on the mother-adolescent dataset shows interpretable trends related to the mother's depression symptoms.

翻译：个性化预测是一种机器学习方法，它依据个体过去的标注观测值预测其未来观测值，通常用于序列任务，例如预测每日情绪评分。在进行个性化预测时，模型可结合两类趋势：(a) 人群间共享的趋势，即人群通用趋势，例如周末更快乐；(b) 每个个体独有的趋势，即个体特异趋势，例如每周有压力的会议。混合效应模型是一种流行的统计模型，通过结合人群通用参数和个体特异参数来研究这两类趋势。尽管线性混合效应模型通过将其与神经网络集成而逐渐在机器学习领域普及，但这些集成目前局限于线性个体特异参数：排除了非线性个体特异趋势。在本文中，我们提出神经混合效应（NME）模型，以可扩展的方式优化神经网络中任意位置的非线性个体特异参数。NME将神经网络优化的效率与非线性混合效应建模相结合。实验结果表明，NME在六个单模态和多模态数据集上提升了性能，包括一个用于预测每日情绪的智能手机数据集，以及一个用于预测情感状态序列的母子数据集（其中一半母亲至少表现出中度抑郁症状）。此外，我们评估了NME在两种模型架构上的表现，包括用于预测情感状态序列的神经条件随机场（CRF），其中CRF学习了情感状态之间的非线性个体特异时间转移。在母子数据集上对这些个体特异转移的分析显示了与母亲抑郁症状相关的可解释趋势。