Multimodal Sentiment Analysis (MSA) has been a popular topic in natural language processing nowadays, at both sentence and aspect level. However, the existing approaches almost require large-size labeled datasets, which bring about large consumption of time and resources. Therefore, it is practical to explore the method for few-shot sentiment analysis in cross-modalities. Previous works generally execute on textual modality, using the prompt-based methods, mainly two types: hand-crafted prompts and learnable prompts. The existing approach in few-shot multi-modality sentiment analysis task has utilized both methods, separately. We further design a hybrid pattern that can combine one or more fixed hand-crafted prompts and learnable prompts and utilize the attention mechanisms to optimize the prompt encoder. The experiments on both sentence-level and aspect-level datasets prove that we get a significant outperformance.

翻译：多模态情感分析（MSA）已成为自然语言处理领域的热门话题，涵盖句子级和方面级两个层面。然而，现有方法几乎都需要大规模标注数据集，这带来了巨大的时间和资源消耗。因此，探索跨模态少样本情感分析的方法具有现实意义。以往研究主要在文本模态上执行，采用基于提示的方法，主要包括两类：手工设计的提示和可学习的提示。现有少样本多模态情感分析任务的方法分别使用了这两种技术。我们进一步设计了一种混合模式，能够结合一个或多个固定手工提示与可学习提示，并利用注意力机制优化提示编码器。在句子级和方面级数据集上的实验证明，我们取得了显著的性能提升。