Large language models (LLMs) have recently gained significant attention due to their unparalleled ability to perform various natural language processing tasks. These models, benefiting from their advanced natural language understanding capabilities, have demonstrated impressive zero-shot performance. However, the pre-training data utilized in LLMs is often confined to a specific corpus, resulting in inherent freshness and temporal scope limitations. Consequently, this raises concerns regarding the effectiveness of LLMs for tasks involving temporal intents. In this study, we aim to investigate the underlying limitations of general-purpose LLMs when deployed for tasks that require a temporal understanding. We pay particular attention to handling factual temporal knowledge through three popular temporal QA datasets. Specifically, we observe low performance on detailed questions about the past and, surprisingly, for rather new information. In manual and automatic testing, we find multiple temporal errors and characterize the conditions under which QA performance deteriorates. Our analysis contributes to understanding LLM limitations and offers valuable insights into developing future models that can better cater to the demands of temporally-oriented tasks. The code is available\footnote{https://github.com/jwallat/temporalblindspots}.

翻译：大型语言模型（LLMs）近期因其在各类自然语言处理任务中展现出的卓越能力而备受关注。这些模型凭借其先进的自然语言理解能力，在零样本任务中表现出色。然而，LLMs所采用的预训练数据通常局限于特定语料库，导致其存在固有的时效性和时间范围局限性。这引发了对LLMs在处理涉及时间意图任务时有效性的担忧。本研究旨在探究通用型LLMs在需要时间理解的任务中存在的根本局限性，并通过三个主流时间问答数据集，重点关注模型对事实性时间知识的处理能力。具体而言，我们观察到模型在处理关于过去事件的详细问题时表现欠佳，令人意外的是，对于较新信息的处理同样存在低效现象。通过人工与自动测试，我们发现了多种时间性错误，并刻画了问答性能下降的条件。本分析有助于理解LLMs的局限性，并为开发未来能更好满足时间导向任务需求的模型提供宝贵见解。相关代码已开源\footnote{https://github.com/jwallat/temporalblindspots}。