In collaborative tasks, effective communication is crucial for achieving joint goals. One such task is collaborative building where builders must communicate with each other to construct desired structures in a simulated environment such as Minecraft. We aim to develop an intelligent builder agent to build structures based on user input through dialogue. However, in collaborative building, builders may encounter situations that are difficult to interpret based on the available information and instructions, leading to ambiguity. In the NeurIPS 2022 Competition NLP Task, we address two key research questions, with the goal of filling this gap: when should the agent ask for clarification, and what clarification questions should it ask? We move towards this target with two sub-tasks, a classification task and a ranking task. For the classification task, the goal is to determine whether the agent should ask for clarification based on the current world state and dialogue history. For the ranking task, the goal is to rank the relevant clarification questions from a pool of candidates. In this report, we briefly introduce our methods for the classification and ranking task. For the classification task, our model achieves an F1 score of 0.757, which placed the 3rd on the leaderboard. For the ranking task, our model achieves about 0.38 for Mean Reciprocal Rank by extending the traditional ranking model. Lastly, we discuss various neural approaches for the ranking task and future direction.

翻译：在协作任务中，有效沟通对达成共同目标至关重要。协作建造是其中的典型任务：建造者需在Minecraft等模拟环境中相互交流，以构建指定结构。我们旨在开发一种基于对话接收用户输入、自主建造结构的智能体。然而在协作建造过程中，建造者可能遇到因现有信息与指令难以理解而导致歧义的情况。为填补这一空白，我们在NeurIPS 2022竞赛NLP任务中聚焦两个核心研究问题：智能体何时应当请求澄清，以及应提出哪些澄清性问题？针对该目标，我们设置了两个子任务：分类任务与排序任务。分类任务的目标是根据当前世界状态与对话历史判断智能体是否需要请求澄清；排序任务的目标则是从候选问题池中对相关澄清性问题进行排序。本报告简要介绍了我们针对分类与排序任务的方法。在分类任务中，模型F1分数达0.757，位列排行榜第三名；在排序任务中，通过扩展传统排序模型，平均倒数排名（Mean Reciprocal Rank）得分约0.38。最后，我们探讨了用于排序任务的多类神经方法及未来研究方向。