General-purpose Large Language Models (LLMs) show significant potential in recruitment applications, where decisions require reasoning over unstructured text, balancing multiple criteria, and inferring fit and competence from indirect productivity signals. Yet, it is still uncertain how LLMs assign importance to each attribute and whether such assignments are in line with economic principles, recruiter preferences or broader societal norms. We propose a framework to evaluate an LLM's decision logic in recruitment, by drawing on established economic methodologies for analyzing human hiring behavior. We build synthetic datasets from real freelancer profiles and project descriptions from a major European online freelance marketplace and apply a full factorial design to estimate how a LLM weighs different match-relevant criteria when evaluating freelancer-project fit. We identify which attributes the LLM prioritizes and analyze how these weights vary across project contexts and demographic subgroups. Finally, we explain how a comparable experimental setup could be implemented with human recruiters to assess alignment between model and human decisions. Our findings reveal that the LLM weighs core productivity signals, such as skills and experience, but interprets certain features beyond their explicit matching value. While showing minimal average discrimination against minority groups, intersectional effects reveal that productivity signals carry different weights between demographic groups.

翻译：通用大型语言模型（LLMs）在招聘应用中展现出巨大潜力，其决策需基于非结构化文本进行推理、平衡多重标准，并从间接生产力信号中推断适配性与胜任力。然而，LLMs如何赋予各属性重要性权重，以及此类赋权是否符合经济学原理、招聘者偏好或更广泛的社会规范，目前仍不明确。本文借鉴分析人类招聘行为的成熟经济学方法，提出一个评估LLM招聘决策逻辑的框架。我们基于欧洲主要在线自由职业者市场的真实自由职业者档案与项目描述构建合成数据集，采用全因子设计方法，量化LLM在评估自由职业者与项目匹配度时对不同匹配相关标准的权衡机制。我们识别了LLM优先考量的属性，并分析了这些权重如何随项目情境与人口统计子群体变化。最后，我们阐释了如何通过可类比实验设置与人类招聘者进行对比，以评估模型决策与人类决策的一致性。研究结果表明，LLM虽能权衡核心生产力信号（如技能与经验），但对某些特征的解释会超越其显性匹配价值。尽管模型对少数群体的平均歧视程度极低，但交叉效应揭示出生产力信号在不同人口统计群体间具有差异化权重。