General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA's multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4-3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized "evidence engine"; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for "ecosystem-aware" Humble AI.
翻译:通用大语言模型对发展领域和政策专家存在错误信息风险,缺乏对可验证输出的认知谦逊性。我们推出AVA(AI+验证分析)——一个基于包含4000余份世界银行报告的多语种策展语料库构建的生成式AI平台。其多智能体流水线使用户能够查询并获取基于证据的综合分析。该平台通过两种机制实现认知谦逊性:引文可验证性(将主张追溯至来源)和合理弃权(对无依据查询予以拒绝并提供理由与引导)。我们通过对来自116个国家不同组织和角色的2200余人进行日志分析、问卷调查及20场访谈开展实地评估。双重差分估计表明,持续使用与每周节省2.4-3.9小时存在关联。定性分析显示,参与者将AVA用作专业化"证据引擎";合理弃权机制明确了边界范围,可信度通过机构来源和锚定页面的引文得到校准。我们贡献了专业化AI的设计指南,并阐明了"生态系统感知型"谦逊AI的发展愿景。