Wearable movement data is collected by nearly all commercially available smartwatches and is a valuable resource for mental health research, reflecting fine-grained temporal behavioral trends. Despite its promise, the development of foundation models for health wearable modeling remains limited when compared to clinical image and text analysis. We designed transformers with patch embeddings and used self-supervised masked autoencoder pretraining on minute-level week-long actigraphy (physical activity intensity measurement) sequences to develop and evaluate the Pretrained Actigraphy Transformer (PAT). PAT is an open-source foundation model for wearable movement time series that combines week-long temporal modeling, psychiatric outcome evaluation, and reproducibility on public data. Pretrained on data from 21,538 U.S. participants in a nationally representative cohort from the National Health and Nutrition Examination Survey (NHANES), PAT consistently outperformed non-foundation-model baselines across mental health prediction tasks-including benzodiazepine and SSRI use, depression, and sleep abnormalities. During the benzodiazepine medication usage prediction task, PAT demonstrated the largest improvement over non-foundational deep learning models commonly used for time-series modeling (i.e., 55.6% improvement over the LSTM, 21.4% improvement over the 1-D CNN, 14.8% improvement over the ConvLSTM). Beyond predictive accuracy, PAT provides interpretable attention maps highlighting specific periods of daily activity most important for clinical predictions, offering model transparency and potential clinical insights. The results suggest that PAT offers an easy-to-deploy, adaptable and scalable solution to advance clinical insight from wearable sensor data for researchers and clinicians. GitHub: https://github.com/njacobsonlab/Pretrained-Actigraphy-Transformer/

翻译：可穿戴运动数据几乎被所有商用智能手表采集，是心理健康研究的宝贵资源，能够反映精细的时间行为趋势。尽管前景广阔，但与临床图像和文本分析相比，用于健康可穿戴建模的基础模型开发仍然有限。我们设计了带有补丁嵌入的Transformer，并在分钟级、周长的活动记录（体力活动强度测量）序列上使用自监督掩码自编码器预训练，开发并评估了预训练活动记录Transformer（PAT）。PAT是一个面向可穿戴运动时间序列的开源基础模型，结合了周长时间建模、精神科结局评估以及在公开数据上的可重复性。基于来自美国国家健康与营养调查（NHANES）具有全国代表性的21,538名美国参与者数据预训练，PAT在心理健康预测任务（包括苯二氮䓬类药物和SSRI的使用、抑郁症及睡眠异常）中始终优于非基础模型基线。在苯二氮䓬类药物使用预测任务中，PAT与常用于时间序列建模的非基础深度学习模型相比，表现出最大改进（即较LSTM改进55.6%，较一维CNN改进21.4%，较ConvLSTM改进14.8%）。除预测准确性外，PAT还提供可解释的注意力图，突出显示对临床预测最重要的每日活动特定时间段，从而提供模型透明性和潜在临床洞见。结果表明，PAT为研究人员和临床医生提供了一种易于部署、可适应且可扩展的解决方案，以从可穿戴传感器数据中推进临床洞见。GitHub：https://github.com/njacobsonlab/Pretrained-Actigraphy-Transformer/