Critical Infrastructure Facilities (CIFs), such as healthcare and transportation facilities, are vital for the functioning of a community, especially during large-scale emergencies. In this paper, we explore a potential application of Large Language Models (LLMs) to monitor the status of CIFs affected by natural disasters through information disseminated in social media networks. To this end, we analyze social media data from two disaster events in two different countries to identify reported impacts to CIFs as well as their impact severity and operational status. We employ state-of-the-art open-source LLMs to perform computational tasks including retrieval, classification, and inference, all in a zero-shot setting. Through extensive experimentation, we report the results of these tasks using standard evaluation metrics and reveal insights into the strengths and weaknesses of LLMs. We note that although LLMs perform well in classification tasks, they encounter challenges with inference tasks, especially when the context/prompt is complex and lengthy. Additionally, we outline various potential directions for future exploration that can be beneficial during the initial adoption phase of LLMs for disaster response tasks.
翻译:关键基础设施设施(CIF),例如医疗和交通设施,对于社区的正常运行至关重要,尤其是在大规模紧急事件期间。本文探索了大语言模型(LLMs)的一种潜在应用:通过社交媒体网络中传播的信息,监测受自然灾害影响的关键基础设施设施的状态。为此,我们分析了来自两个不同国家灾难事件中的社交媒体数据,以识别对关键基础设施设施造成的影响报告,以及其影响严重程度和运行状态。我们采用最先进的开源大语言模型来执行计算任务,包括检索、分类和推理,全部在零样本设定下完成。通过大量实验,我们使用标准评估指标报告了这些任务的结果,并揭示了关于大语言模型优缺点的洞见。我们注意到,尽管大语言模型在分类任务中表现良好,但在推理任务中遇到挑战,尤其是当上下文/提示复杂且冗长时。此外,我们概述了未来探索的各种潜在方向,这些方向在LLM初期应用于灾害响应任务阶段将具有实用价值。