This position paper argues that job exposure to AI should be measured with grounded, evidence-based methods, not inferred from LLM priors alone. Current theoretical exposure measures use zero-shot prompting to classify task-level AI exposure, generating labels with no explicit evidence, no transparent chain of reasoning, and no external validation. The stakes of these measurements are too high to rely on such methods, as they influence policy making, where public and private funds are directed, and how workers understand their future prospects. We therefore argue that AI capability claims should meet three standards: reproducibility, external grounding, and inspectability. We propose a retrieval-augmented framework that assigns AI exposure labels to all 18,796 occupation--task pairs in O*NET 30.2, using open-weight reasoning and instruct models with retrieved news articles and academic paper abstracts as evidence of current AI capabilities. Relative to a zero-shot baseline, the grounded condition is preferred in over 72\% of disagreement cases under both automatic and human evaluation, and yields scores that align more closely with observed real-world AI usage. Taken together, these findings suggest that evidence-grounded measurement better captures what current AI systems can plausibly do in practice, rather than what a model asserts without external evidence. Because AI capabilities continue to change, the measurements used to inform policy must evolve with them: theoretical AI exposure scores should be periodically reassessed, not inherited as immutable ground truth.
翻译:这篇立场论文主张,工作岗位的AI暴露度应采用基于实证的方法进行衡量,而非仅从大语言模型的先验知识中推断。现有的理论暴露度量方法通过零样本提示对任务级AI暴露度进行分类,生成的标签既缺乏明确的证据支撑,又无透明的推理链条,且未经外部验证。鉴于这些度量结果对政策制定、公共与私人资金流向以及劳动者对未来职业前景的认知具有重大影响,其重要性不容依赖此类方法。为此,我们主张AI能力的主张应满足三项标准:可复现性、外部可验证性与可审查性。我们提出一个检索增强框架,为O*NET 30.2中全部18,796个职业-任务对分配AI暴露度标签,该框架利用开源推理与指令模型,并检索新闻文章与学术论文摘要作为当前AI能力的实证依据。相较于零样本基线,在自动与人工评估中,超过72%的争议案例更倾向于采用基于实证的条件,且由此得出的分数与观察到的现实世界AI使用情况更为吻合。综合来看,这些发现表明,基于实证的衡量方法更能反映当前AI系统在实际中可能实现的能力,而非模型仅凭外部证据断言的结果。由于AI能力持续演进,为政策提供依据的度量方法也应随之发展:理论上的AI暴露度分数应定期重新评估,而非视为不可更改的绝对真理。