In this technical report, we present our solution for the EgoPlan Challenge in ICML 2024. To address the real-world egocentric task planning problem, we introduce a novel planning framework which comprises three stages: long-term memory Extraction, context-awared Planning, and multi-iteration Decision, named EPD. Given the task goal, task progress, and current observation, the extraction model first extracts task-relevant memory information from the progress video, transforming the complex long video into summarized memory information. The planning model then combines the context of the memory information with fine-grained visual information from the current observation to predict the next action. Finally, through multi-iteration decision-making, the decision model comprehensively understands the task situation and current state to make the most realistic planning decision. On the EgoPlan-Test set, EPD achieves a planning accuracy of 53.85% over 1,584 egocentric task planning questions. We have made all codes available at https://github.com/Kkskkkskr/EPD .

翻译：在本技术报告中，我们介绍了针对ICML 2024 EgoPlan挑战赛的解决方案。为解决现实世界中以自我为中心的任务规划问题，我们提出了一种新颖的三阶段规划框架：长期记忆提取、上下文感知规划与多轮决策，命名为EPD。给定任务目标、任务进度和当前观察，提取模型首先从进度视频中提取与任务相关的记忆信息，将复杂的长视频转化为概括性的记忆信息。规划模型随后将记忆信息的上下文与当前观察中的细粒度视觉信息相结合，以预测下一个动作。最后，通过多轮决策过程，决策模型全面理解任务情境与当前状态，做出最符合现实的规划决策。在EgoPlan-Test数据集上，EPD在1,584个以自我为中心的任务规划问题上实现了53.85%的规划准确率。我们已将全部代码开源，地址为 https://github.com/Kkskkkskr/EPD 。