Recognizing vulnerability is crucial for understanding and implementing targeted support to empower individuals in need. This is especially important at the European Court of Human Rights (ECtHR), where the court adapts Convention standards to meet actual individual needs and thus ensures effective human rights protection. However, the concept of vulnerability remains elusive at the ECtHR and no prior NLP research has dealt with it. To enable future research in this area, we present VECHR, a novel expert-annotated multi-label dataset comprising of vulnerability type classification and explanation rationale. We benchmark the performance of state-of-the-art models on VECHR from both prediction and explainability perspectives. Our results demonstrate the challenging nature of the task with lower prediction performance and limited agreement between models and experts. Further, we analyze the robustness of these models in dealing with out-of-domain (OOD) data and observe overall limited performance. Our dataset poses unique challenges offering significant room for improvement regarding performance, explainability, and robustness.

翻译：识别脆弱性对于理解和实施针对性支持以赋权有需要的个体至关重要。这在欧洲人权法院（ECtHR）中尤为重要，该法院会根据实际个体需求调整《公约》标准，从而确保有效的人权保护。然而，脆弱性概念在ECtHR中仍难以界定，且此前尚无自然语言处理研究涉足该领域。为推动该领域的未来研究，我们提出了VECHR——一个由专家标注的新型多标签数据集，包含脆弱性类型分类结果及其解释依据。我们从预测与可解释性两个维度，测算了先进模型在VECHR上的性能表现。研究结果表明，该任务具有挑战性：预测性能较低，模型与专家的解释一致性有限。此外，我们分析了这些模型处理域外数据时的稳健性，观察到整体表现有限。我们的数据集在性能、可解释性与稳健性方面设置了独特挑战，为后续改进提供了广阔空间。