In the face of climate change, are companies really taking substantial steps toward more sustainable operations? A comprehensive answer lies in the dense, information-rich landscape of corporate sustainability reports. However, the sheer volume and complexity of these reports make human analysis very costly. Therefore, only a few entities worldwide have the resources to analyze these reports at scale, which leads to a lack of transparency in sustainability reporting. Empowering stakeholders with LLM-based automatic analysis tools can be a promising way to democratize sustainability report analysis. However, developing such tools is challenging due to (1) the hallucination of LLMs and (2) the inefficiency of bringing domain experts into the AI development loop. In this paper, we ChatReport, a novel LLM-based system to automate the analysis of corporate sustainability reports, addressing existing challenges by (1) making the answers traceable to reduce the harm of hallucination and (2) actively involving domain experts in the development loop. We make our methodology, annotated datasets, and generated analyses of 1015 reports publicly available.

翻译：面对气候变化，企业是否真正采取了实质性措施以实现更可持续的运营？这一问题的全面答案深藏于信息密集的企业可持续性报告中。然而，这些报告的庞大体量和复杂程度使得人工分析成本极为高昂。因此，全球仅有少数机构具备规模化分析此类报告的资源，导致可持续性报告缺乏透明度。通过基于大语言模型的自动分析工具赋能利益相关者，或可成为推动可持续性报告分析民主化的有效途径。然而，开发此类工具面临两大挑战：（1）大语言模型的幻觉问题；（2）将领域专家纳入人工智能开发循环的效率低下。本文提出ChatReport——一种基于大语言模型的新型系统，通过（1）使答案可溯源以降低幻觉危害，以及（2）积极让领域专家参与开发循环，实现了企业可持续性报告的自动化分析。我们公开了研究方法、标注数据集以及对1015份报告的分析结果。