Large Language Models (LLMs) have transformed artificial intelligence by excelling in complex natural language processing tasks. Their ability to generate human-like text has opened new possibilities for market research, particularly in conjoint analysis, where understanding consumer preferences is essential but often resource-intensive. Traditional survey-based methods face limitations in scalability and cost, making LLM-generated data a promising alternative. However, while LLMs have the potential to simulate real consumer behavior, recent studies highlight a significant gap between LLM-generated and human data, with biases introduced when substituting between the two. In this paper, we address this gap by proposing a novel statistical data augmentation approach that efficiently integrates LLM-generated data with real data in conjoint analysis. This results in statistically robust estimators with consistent and asymptotically normal properties, in contrast to naive approaches that simply substitute human data with LLM-generated data, which can exacerbate bias. We further present a finite-sample performance bound on the estimation error. We validate our framework through an empirical study on COVID-19 vaccine preferences, demonstrating its superior ability to reduce estimation error and save data and costs by 24.9% to 79.8%. In contrast, naive approaches fail to save data due to the inherent biases in LLM-generated data compared to human data. Another empirical study on sports car choices validates the robustness of our results. Our findings suggest that while LLM-generated data is not a direct substitute for human responses, it can serve as a valuable complement when used within a robust statistical framework.

翻译：大型语言模型（LLMs）通过擅长处理复杂的自然语言处理任务，已彻底改变了人工智能领域。其生成类人文本的能力为市场研究开辟了新的可能性，尤其是在联合分析中——理解消费者偏好至关重要但通常资源密集。传统的基于调查的方法在可扩展性和成本方面存在局限性，这使得LLM生成的数据成为一种有前景的替代方案。然而，尽管LLMs有潜力模拟真实的消费者行为，但近期研究凸显出LLM生成数据与人类数据之间存在显著差距，且当两者相互替代时会引入偏差。在本文中，我们通过提出一种新颖的统计数据增强方法来弥合这一差距，该方法在联合分析中高效整合了LLM生成的数据与真实数据。与简单用LLM生成数据替代人类数据（这可能会加剧偏差）的朴素方法不同，我们提出的方法可生成具有一致性和渐近正态性的统计稳健估计量。我们还给出了估计误差的有限样本性能界。我们通过一项关于COVID-19疫苗偏好的实证研究验证了该框架，证明其在降低估计误差方面具有优越性，可节省24.9%至79.8%的数据与成本。相比之下，由于LLM生成数据与人类数据相比存在固有偏差，朴素方法无法节省数据。另一项关于跑车选择的实证研究验证了我们结果的稳健性。我们的研究结果表明，LLM生成的数据虽不能直接替代人类响应，但在稳健统计框架内可作为有价值的补充。