Language models produce authoritative, persuasive responses even when those responses rest on fabricated expertise. Measuring this fabrication propensity directly across all domains is intractable, but AI identity disclosure provides a clean test: when a model assigned a professional persona is asked about its expertise origins, it can either disclose its AI nature or fabricate a human professional history. Because the ground truth is known-the model is not a neurosurgeon-non-disclosure constitutes unambiguous fabrication. Using a factorial evaluation design, sixteen open-weight models (4B-671B parameters) were audited under identical conditions across 19,200 trials. Under professional personas-neurosurgeon, financial advisor, classical musician-models that disclose their AI nature in 99.8-99.9% of interactions under neutral conditions instead fabricated professional credentials, training narratives, and embodied experiences. Fabrication rates varied unpredictably: a 14B model disclosed in 61.4% of interactions while a 70B model disclosed in just 4.1%. Domain-specific inconsistency was pronounced: a Financial Advisor persona elicited 35.2% disclosure at the first prompt while a Neurosurgeon persona elicited only 3.6%-a 9.7-fold difference. Model identity provided substantially larger improvement in fitting observations than parameter count (Delta R_adj^2 = 0.375 vs 0.012). An additional experiment found that adding explicit disclosure permission to persona system prompts increased disclosure from 23.7% to 65.8%, indicating that honest self-representation is a suppressed default rather than an absent capability-models can disclose but do not when persona instructions are silent on self-disclosure. The propensity to fabricate expertise is context-dependent rather than a stable model property, requiring deliberate behavior design and domain-specific verification.
翻译:暂无翻译