Recent advances in large language models (LLMs) have enabled the creation of highly effective chatbots. However, the compute costs of widely deploying LLMs have raised questions about profitability. Companies have proposed exploring ad-based revenue streams for monetizing LLMs, which could serve as the new de facto platform for advertising. This paper investigates the implications of personalizing LLM advertisements to individual users via a between-subjects experiment with 179 participants. We developed a chatbot that embeds personalized product advertisements within LLM responses, inspired by similar forays by AI companies. The evaluation of our benchmarks showed that ad injection only slightly impacted LLM performance, particularly response desirability. Results revealed that participants struggled to detect ads, and even preferred LLM responses with hidden advertisements. Rather than clicking on our advertising disclosure, participants tried changing their advertising settings using natural language queries. We created an advertising dataset and an open-source LLM, Phi-4-Ads, fine-tuned to serve ads and flexibly adapt to user preferences.

翻译：近年来，大型语言模型（LLM）的进展使得创建高效聊天机器人成为可能。然而，大规模部署LLM的计算成本引发了关于盈利能力的疑问。企业已提出探索基于广告的收入流以实现LLM的货币化，这可能成为广告投放的新事实平台。本文通过一项涉及179名参与者的组间实验，研究了向个体用户个性化推送LLM广告的影响。受人工智能公司类似尝试的启发，我们开发了一个在LLM回复中嵌入个性化产品广告的聊天机器人。对基准测试的评估表明，广告注入仅轻微影响LLM性能，尤其是回复的合意性。结果显示，参与者难以察觉广告，甚至更偏好包含隐藏广告的LLM回复。参与者并未点击我们的广告披露声明，而是尝试使用自然语言查询来更改广告设置。我们创建了一个广告数据集和一个开源LLM——Phi-4-Ads，该模型经过微调以投放广告并灵活适应用户偏好。