We consider dyadic zero-shot event extraction (EE) to identify actions between pairs of actors. The \emph{zero-shot} setting allows social scientists or other non-computational researchers to extract any customized, user-specified set of events without training, resulting in a \emph{dyadic} event database, allowing insight into sociopolitical relational dynamics among actors and the higher level organizations or countries they represent. Unfortunately, we find that current zero-shot EE methods perform poorly for the task, with issues including word sense ambiguity, modality mismatch, and efficiency. Straightforward application of large language model prompting typically performs even worse. We address these challenges with a new fine-grained, multi-stage generative question-answer method, using a Monte Carlo approach to exploit and overcome the randomness of generative outputs. It performs 90\% fewer queries than a previous approach, with strong performance on the widely-used Automatic Content Extraction dataset. Finally, we extend our method to extract affiliations of actor arguments and demonstrate our method and findings on a dyadic international relations case study.
翻译:我们考虑二元零样本事件抽取(EE)来识别成对行动者之间的动作。零样本设置允许社会科学家或其他非计算研究人员无需训练即可提取任意定制、用户指定的事件集,从而构建二元事件数据库,为洞察行动者及其所代表的更高层组织或国家之间的社会政治关系动态提供依据。然而,我们发现当前的零样本事件抽取方法在该任务上表现不佳,存在词汇歧义、模态不匹配和效率低下等问题。直接应用大型语言模型提示的方法通常表现更差。我们通过一种新的细粒度、多阶段生成式问答方法来解决这些挑战,利用蒙特卡洛方法利用并克服生成输出的随机性。与先前方法相比,该方法减少了90%的查询次数,并在广泛使用的自动内容抽取数据集上表现出色。最后,我们将方法扩展到抽取行动者参数的隶属关系,并通过一个二元国际关系案例研究展示我们的方法和发现。