The reliability of large language models (LLMs) in production environments remains significantly constrained by their propensity to generate hallucinations -- fluent, plausible-sounding outputs that contradict or fabricate information. While hallucination detection has recently emerged as a priority in English-centric benchmarks, low-to-medium resource languages such as Vietnamese remain inadequately covered by standardized evaluation frameworks. This paper introduces the DSC2025 ViHallu Challenge, the first large-scale shared task for detecting hallucinations in Vietnamese LLMs. We present the ViHallu dataset, comprising 10,000 annotated triplets of (context, prompt, response) samples systematically partitioned into three hallucination categories: no hallucination, intrinsic, and extrinsic hallucinations. The dataset incorporates three prompt types -- factual, noisy, and adversarial -- to stress-test model robustness. A total of 111 teams participated, with the best-performing system achieving a macro-F1 score of 84.80\%, compared to a baseline encoder-only score of 32.83\%, demonstrating that instruction-tuned LLMs with structured prompting and ensemble strategies substantially outperform generic architectures. However, the gap to perfect performance indicates that hallucination detection remains a challenging problem, particularly for intrinsic (contradiction-based) hallucinations. This work establishes a rigorous benchmark and explores a diverse range of detection methodologies, providing a foundation for future research into the trustworthiness and reliability of Vietnamese language AI systems.
翻译:在生产环境中,大语言模型(LLMs)的可靠性仍然因其产生幻觉的倾向而受到显著制约——幻觉是指那些流畅、听起来合理但与事实相矛盾或捏造信息的输出。尽管幻觉检测最近已成为以英语为中心的基准测试中的优先事项,但像越南语这样的中低资源语言,仍然缺乏标准化的评估框架来充分覆盖。本文介绍了DSC2025 ViHallu挑战赛,这是首个用于检测越南语LLMs幻觉的大规模共享任务。我们提出了ViHallu数据集,包含10,000个标注的(上下文,提示,响应)三元组样本,系统性地划分为三类幻觉:无幻觉、内在幻觉和外在幻觉。该数据集融合了三种提示类型——事实性、噪声性和对抗性——以压力测试模型的鲁棒性。共有111支队伍参与,表现最佳的系统获得了84.80%的宏平均F1分数,而仅使用编码器的基线模型分数为32.83%,这表明采用结构化提示和集成策略的指令调优LLMs显著优于通用架构。然而,与完美性能之间的差距表明,幻觉检测仍然是一个具有挑战性的问题,特别是对于内在(基于矛盾的)幻觉。这项工作建立了一个严格的基准,并探索了多样化的检测方法,为未来研究越南语人工智能系统的可信度和可靠性奠定了基础。