Multimodal Sentiment Analysis (MSA) has been a popular topic in natural language processing nowadays, at both sentence and aspect level. However, the existing approaches almost require large-size labeled datasets, which bring about large consumption of time and resources. Therefore, it is practical to explore the method for few-shot sentiment analysis in cross-modalities. Previous works generally execute on textual modality, using the prompt-based methods, mainly two types: hand-crafted prompts and learnable prompts. The existing approach in few-shot multi-modality sentiment analysis task has utilized both methods, separately. We further design a hybrid pattern that can combine one or more fixed hand-crafted prompts and learnable prompts and utilize the attention mechanisms to optimize the prompt encoder. The experiments on both sentence-level and aspect-level datasets prove that we get a significant outperformance.

翻译：多模态情感分析是自然语言处理领域的热门话题，涵盖句子级和方面级两个层面。然而，现有方法几乎都需要大规模标注数据集，导致时间和资源消耗巨大。因此，探索跨模态少样本情感分析方法具有实际意义。以往研究主要针对文本模态，采用提示方法，主要分为两类：手工构建提示和可学习提示。现有少样本多模态情感分析任务的方法已分别采用这两种方法。我们进一步设计了一种混合模式，能够结合一个或多个固定手工构建提示与可学习提示，并利用注意力机制优化提示编码器。在句子级和方面级数据集上的实验证明，我们取得了显著的性能提升。