Is it possible accurately classify political relations within evolving event ontologies without extensive annotations? This study investigates zero-shot learning methods that use expert knowledge from existing annotation codebook, and evaluates the performance of advanced ChatGPT (GPT-3.5/4) and a natural language inference (NLI)-based model called ZSP. ChatGPT uses codebook's labeled summaries as prompts, whereas ZSP breaks down the classification task into context, event mode, and class disambiguation to refine task-specific hypotheses. This decomposition enhances interpretability, efficiency, and adaptability to schema changes. The experiments reveal ChatGPT's strengths and limitations, and crucially show ZSP's outperformance of dictionary-based methods and its competitive edge over some supervised models. These findings affirm the value of ZSP for validating event records and advancing ontology development. Our study underscores the efficacy of leveraging transfer learning and existing domain expertise to enhance research efficiency and scalability.
翻译:能否在不依赖大量标注的情况下,准确地对演化中的事件本体中的政治关系进行分类?本研究探索了利用现有标注代码本中的专家知识进行零样本学习的方法,并评估了先进的ChatGPT(GPT-3.5/4)与一种基于自然语言推理(NLI)的模型ZSP的性能。ChatGPT使用代码本中带标签的摘要作为提示,而ZSP则将分类任务分解为上下文理解、事件模式识别和类别消歧,以优化任务特定的假设。这种分解增强了模型的可解释性、效率以及对模式变化的适应能力。实验揭示了ChatGPT的优势与局限,并关键性地表明ZSP在性能上超越了基于词典的方法,甚至在某些监督模型面前也展现出竞争力。这些发现肯定了ZSP在验证事件记录和推动本体发展方面的价值。我们的研究强调了利用迁移学习和现有领域专业知识来提升研究效率与可扩展性的有效性。