In today's fast-paced world, the rates of stress and depression present a surge. Social media provide assistance for the early detection of mental health conditions. Existing methods mainly introduce feature extraction approaches and train shallow machine learning classifiers. Other researches use deep neural networks or transformers. Despite the fact that transformer-based models achieve noticeable improvements, they cannot often capture rich factual knowledge. Although there have been proposed a number of studies aiming to enhance the pretrained transformer-based models with extra information or additional modalities, no prior work has exploited these modifications for detecting stress and depression through social media. In addition, although the reliability of a machine learning model's confidence in its predictions is critical for high-risk applications, there is no prior work taken into consideration the model calibration. To resolve the above issues, we present the first study in the task of depression and stress detection in social media, which injects extra linguistic information in transformer-based models, namely BERT and MentalBERT. Specifically, the proposed approach employs a Multimodal Adaptation Gate for creating the combined embeddings, which are given as input to a BERT (or MentalBERT) model. For taking into account the model calibration, we apply label smoothing. We test our proposed approaches in three publicly available datasets and demonstrate that the integration of linguistic features into transformer-based models presents a surge in the performance. Also, the usage of label smoothing contributes to both the improvement of the model's performance and the calibration of the model. We finally perform a linguistic analysis of the posts and show differences in language between stressful and non-stressful texts, as well as depressive and non-depressive posts.

翻译：在当今快节奏的世界中，压力与抑郁的发病率呈现上升趋势。社交媒体为心理健康状况的早期检测提供了辅助手段。现有方法主要引入特征提取技术并训练浅层机器学习分类器，另一些研究则采用深度神经网络或Transformer。尽管基于Transformer的模型取得了显著改进，但它们往往无法捕捉丰富的事实知识。虽然已有诸多研究致力于通过额外信息或附加模态来增强预训练的Transformer模型，但尚无前期工作利用这些改进手段通过社交媒体检测压力与抑郁。此外，尽管机器学习模型对其预测置信度的可靠性对于高风险应用至关重要，但此前尚无研究考虑模型校准问题。为解决上述问题，我们首次在社交媒体的抑郁与压力检测任务中，向基于Transformer的模型（即BERT和MentalBERT）注入额外的语言信息。具体而言，所提出的方法采用多模态适应门来生成组合嵌入，并将其作为BERT（或MentalBERT）模型的输入。为考虑模型校准，我们应用标签平滑技术。我们在三个公开数据集上测试所提出的方法，并证明将语言特征集成到基于Transformer的模型中能显著提升性能。同时，标签平滑的使用有助于改善模型性能并提升校准效果。最后，我们对帖文进行语言学分析，展示压力与非压力文本、抑郁与非抑郁帖文之间的语言差异。