Fine-grained opinion analysis of text provides a detailed understanding of expressed sentiments, including the addressed entity. Although this level of detail is sound, it requires considerable human effort and substantial cost to annotate opinions in datasets for training models, especially across diverse domains and real-world applications. We explore the feasibility of LLMs as automatic annotators for fine-grained opinion analysis, addressing the shortage of domain-specific labelled datasets. In this work, we use a declarative annotation pipeline. This approach reduces the variability of manual prompt engineering when using LLMs to identify fine-grained opinion spans in text. We also present a novel methodology for an LLM to adjudicate multiple labels and produce final annotations. After trialling the pipeline with models of different sizes for the Aspect Sentiment Triplet Extraction (ASTE) and Aspect-Category-Opinion-Sentiment (ACOS) analysis tasks, we show that LLMs can serve as automatic annotators and adjudicators, achieving high Inter-Annotator Agreement across individual LLM-based annotators. This reduces the cost and human effort needed to create these fine-grained opinion-annotated datasets.

翻译：文本的细粒度观点分析提供了对表达情感的详细理解，包括所涉及的实体。尽管这种细节层次是合理的，但为训练模型所需的数据集标注观点需要大量的人力投入和成本，尤其是在跨领域和现实应用场景中。我们探索了将大型语言模型（LLMs）作为细粒度观点分析的自动标注器的可行性，以解决领域特定标注数据集短缺的问题。在本工作中，我们采用了一种声明式标注流程。该方法减少了在使用LLMs识别文本中细粒度观点跨度时手动提示工程的可变性。我们还提出了一种新颖的方法，使LLM能够对多个标签进行裁决并生成最终标注。通过在方面情感三元组抽取（ASTE）和方面-类别-观点-情感（ACOS）分析任务上对不同规模的模型进行流程试验，我们证明了LLMs可以作为自动标注器和裁决器，在基于LLM的个体标注器之间实现较高的标注者间一致性。这降低了创建此类细粒度观点标注数据集所需的成本和人力投入。