Recommender systems generally optimises user engagement, but this approach is dangerous in mental health contexts. When vulnerable users show signs of suicidal ideation, standard algorithms often trap them in echo chambers of harmful content, worsening their psychological state. In response, we introduce RankAid, a re-ranking method that prioritises clinical safety alongside predictive relevance. It works as an add-on layer to existing models: it penalises risky items and boosts therapeutic content depending on the user's current level of vulnerability. We evaluated this approach using the MovieLens 1M dataset, where items were semantically annotated for clinical risk and therapeutic value using large language models. Our simulations show that our algorithm successfully blocks the recommendation of harmful content during crisis peaks, actively reshaping the feed to support emotional de-escalation. Furthermore, this safety intervention only causes a controlled, acceptable drop in standard accuracy metrics like NDCG. By using asymmetric hyperparameters, RankAid also gives system administrators the flexibility to tune the severity of the intervention based on specific clinical guidelines.

翻译：推荐系统通常优化用户参与度，但在心理健康情境中，这种方法存在危险。当脆弱用户表现出自杀意念迹象时，标准算法往往将其困在有害内容的回音室中，加剧其心理状态恶化。为此，我们提出RankAid——一种将临床安全性与预测相关性置于同等优先级的重排序方法。该方法作为现有模型的附加层运行：根据用户当前的脆弱程度，惩罚高风险内容并强化治疗性内容。我们使用MovieLens 1M数据集进行评估，通过大语言模型对该数据集中项目的临床风险和治疗价值进行语义标注。模拟实验表明，我们的算法能在危机高峰期间成功阻断有害内容的推荐，主动重塑信息流以支持情绪平复。此外，该安全干预仅导致NDCG等标准准确性指标的受控可接受下降。通过使用非对称超参数，RankAid还赋予系统管理员根据特定临床指南调整干预强度的灵活性。