In this paper, we present our solution to a Multi-modal Algorithmic Reasoning Task: SMART-101 Challenge. Different from the traditional visual question-answering datasets, this challenge evaluates the abstraction, deduction, and generalization abilities of neural networks in solving visuolinguistic puzzles designed specifically for children in the 6-8 age group. We employed a divide-and-conquer approach. At the data level, inspired by the challenge paper, we categorized the whole questions into eight types and utilized the llama-2-chat model to directly generate the type for each question in a zero-shot manner. Additionally, we trained a yolov7 model on the icon45 dataset for object detection and combined it with the OCR method to recognize and locate objects and text within the images. At the model level, we utilized the BLIP-2 model and added eight adapters to the image encoder VIT-G to adaptively extract visual features for different question types. We fed the pre-constructed question templates as input and generated answers using the flan-t5-xxl decoder. Under the puzzle splits configuration, we achieved an accuracy score of 26.5 on the validation set and 24.30 on the private test set.

翻译：本文提出了一种针对多模态算法推理任务SMART-101挑战赛的解决方案。与传统视觉问答数据集不同，该挑战旨在评估神经网络在解决专为6-8岁儿童设计的视觉语言谜题时的抽象、演绎和泛化能力。我们采用分治策略。在数据层面，受挑战论文启发，将所有问题归为八类，并利用llama-2-chat模型以零样本方式直接生成每个问题的类别。此外，我们在icon45数据集上训练了yolov7模型进行目标检测，结合OCR方法识别并定位图像中的物体和文本。在模型层面，我们采用BLIP-2模型，在图像编码器VIT-G上添加八个适配器，针对不同问题类型自适应提取视觉特征。将预构建的问题模板作为输入，利用flan-t5-xxl解码器生成答案。在谜题分割配置下，验证集准确率为26.5，私有测试集准确率为24.30。