Using a legally obtained dataset of 34 copyrighted O'Reilly Media books, we apply the DE-COP membership inference attack method to investigate whether OpenAI's large language models show recognition of copyrighted content. Our results based on this small sample suggest that GPT-4o, OpenAI's more recent and capable model, exhibits patterns consistent with recognition of pay-walled book content, with an AUROC score of 0.82 (95% bootstrapped CI: 0.60-0.96), though this wide confidence interval reflects substantial uncertainty due to the limited number of books tested. GPT-4o Mini, as a much smaller model, shows little recognition of any O'Reilly Media content with an AUROC score of 0.56 (0.28-0.83) for non-public data. Testing multiple models, with the same cutoff date, provides a partial control for potential language shifts over time that might bias our findings, though differences in model size, architecture, and potentially training data composition limit the strength of this control. These preliminary results underscore the importance of increased corporate transparency regarding pre-training data sources and the development of formal licensing frameworks for AI content training. Our principal contribution is our examination of public and non public data separately.

翻译：基于合法获取的包含34本O'Reilly Media版权书籍的数据集，我们应用DE-COP成员推理攻击方法，探究OpenAI的大语言模型是否表现出对版权内容的识别能力。基于这一小样本的实验结果表明：OpenAI较新且性能更强的GPT-4o模型呈现出与识别付费书籍内容相一致的规律，其AUROC评分为0.82（95%自助法置信区间：0.60-0.96），不过较宽的置信区间反映了受限于测试书籍数量而存在显著不确定性。作为规模小得多的模型，GPT-4o Mini对O'Reilly Media内容的识别能力极弱，非公共数据的AUROC评分仅为0.56（0.28-0.83）。通过测试具有相同截止日期的多个模型，可部分控制可能造成偏差的历时性语言演变，但模型规模、架构及潜在训练数据构成的差异削弱了这一控制的有效性。这些初步结果凸显了增加企业关于预训练数据源的透明度、建立AI内容训练正式许可框架的重要性。本研究的主要贡献在于分别系统检验了公共与非公共数据的识别特征。