In today's fast-paced world, the rates of stress and depression present a surge. Social media provide assistance for the early detection of mental health conditions. Existing methods mainly introduce feature extraction approaches and train shallow machine learning classifiers. Other researches use deep neural networks or transformers. Despite the fact that transformer-based models achieve noticeable improvements, they cannot often capture rich factual knowledge. Although there have been proposed a number of studies aiming to enhance the pretrained transformer-based models with extra information or additional modalities, no prior work has exploited these modifications for detecting stress and depression through social media. In addition, although the reliability of a machine learning model's confidence in its predictions is critical for high-risk applications, there is no prior work taken into consideration the model calibration. To resolve the above issues, we present the first study in the task of depression and stress detection in social media, which injects extra linguistic information in transformer-based models, namely BERT and MentalBERT. Specifically, the proposed approach employs a Multimodal Adaptation Gate for creating the combined embeddings, which are given as input to a BERT (or MentalBERT) model. For taking into account the model calibration, we apply label smoothing. We test our proposed approaches in three publicly available datasets and demonstrate that the integration of linguistic features into transformer-based models presents a surge in the performance. Also, the usage of label smoothing contributes to both the improvement of the model's performance and the calibration of the model. We finally perform a linguistic analysis of the posts and show differences in language between stressful and non-stressful texts, as well as depressive and non-depressive posts.

翻译：在当今快节奏社会中，压力与抑郁发生率呈现激增态势。社交媒体为心理健康状况的早期检测提供了辅助手段。现有方法主要引入特征提取技术并训练浅层机器学习分类器，其他研究则采用深度神经网络或Transformer模型。尽管基于Transformer的模型取得了显著改进，但它们往往难以捕捉丰富的事实性知识。虽然已有诸多研究致力于通过额外信息或多模态方式增强预训练Transformer模型，但尚无先例将其应用于社交媒体压力与抑郁检测任务。此外，尽管机器学习模型预测置信度的可靠性对于高风险应用至关重要，但现有研究未考虑模型校准问题。为解决上述问题，我们首次在社交媒体抑郁与压力检测任务中，向基于Transformer的模型（即BERT与MentalBERT）注入额外语言学信息。具体而言，所提方法采用多模态自适应门控机制构建联合嵌入表示，并将其作为BERT（或MentalBERT）模型的输入。为兼顾模型校准，我们应用标签平滑技术。在三个公开数据集上的实验表明，将语言学特征融入Transformer模型可显著提升性能表现。同时，标签平滑的使用既改善了模型性能，又促进了模型校准效果。我们最后对帖文进行语言学分析，揭示了压力性与非压力性文本、抑郁性与非抑郁性帖文之间的语言差异。