Task planning for robotic manipulation with large language models (LLMs) is an emerging area. Prior approaches rely on specialized models, fine tuning, or prompt tuning, and often operate in an open loop manner without robust environmental feedback, making them fragile in dynamic settings.MALLVi present a Multi Agent Large Language and Vision framework that enables closed loop feedback driven robotic manipulation. Given a natural language instruction and an image of the environment, MALLVi generates executable atomic actions for a robot manipulator. After action execution, a Vision Language Model (VLM) evaluates environmental feedback and decides whether to repeat the process or proceed to the next step Rather than using a single model, MALLVi coordinates specialized agents, Decomposer, Localizer, Thinker, and Reflector, to manage perception, localization, reasoning, and high level planning. An optional Descriptor agent provides visual memory of the initial state. The Reflector supports targeted error detection and recovery by reactivating only relevant agents, avoiding full replanning.Experiments in simulation and real world settings show that iterative closed loop multi agent coordination improves generalization and increases success rates in zero shot manipulation tasks.Code available at https://github.com/iman1234ahmadi/MALLVI.

翻译：基于大语言模型（LLM）的机器人操作任务规划是一个新兴领域。现有方法依赖于专用模型、微调或提示调优，且通常以开环方式运行，缺乏鲁棒的环境反馈，导致其在动态环境中表现脆弱。MALLVI提出了一种多智能体大语言与视觉框架，实现了基于闭环反馈驱动的机器人操作。给定自然语言指令和环境图像，MALLVI为机器人操作器生成可执行的原子动作。动作执行后，视觉语言模型（VLM）评估环境反馈，并决定重复该过程或进入下一步。MALLVI并非使用单一模型，而是协调分解器、定位器、思考器与反思器四个专用智能体，分别管理感知、定位、推理与高层规划。可选的描述器智能体提供初始状态的视觉记忆。反思器通过仅重新激活相关智能体来支持针对性错误检测与恢复，避免了完全重新规划。仿真与真实环境实验表明，迭代式闭环多智能体协调提升了零样本操作任务的泛化能力与成功率。代码发布于 https://github.com/iman1234ahmadi/MALLVI。