Humans can effortlessly perform very complex, dexterous manipulation tasks by reacting to sensor observations. In contrast, robots can not perform reactive manipulation and they mostly operate in open-loop while interacting with their environment. Consequently, the current manipulation algorithms either are inefficient in performance or can only work in highly structured environments. In this paper, we present closed-loop control of a complex manipulation task where a robot uses a tool to interact with objects. Manipulation using a tool leads to complex kinematics and contact constraints that need to be satisfied for generating feasible manipulation trajectories. We first present an open-loop controller design using Non-Linear Programming (NLP) that satisfies these constraints. In order to design a closed-loop controller, we present a pose estimator of objects and tools using tactile sensors. Using our tactile estimator, we design a closed-loop controller based on Model Predictive Control (MPC). The proposed algorithm is verified using a 6 DoF manipulator on tasks using a variety of objects and tools. We verify that our closed-loop controller can successfully perform tool manipulation under several unexpected contacts. Video summarizing this work and hardware experiments are found https://youtu.be/VsClK04qDhk.

翻译：人类能够通过感知观察轻松完成非常复杂、灵巧的操作任务。相比之下，机器人无法进行反应式操控，在与环境交互时大多采用开环控制。因此，当前的操控算法要么性能低下，要么只能在高度结构化的环境中工作。本文针对机器人使用工具与物体交互的复杂操控任务，提出了闭环控制方法。使用工具进行操控会产生复杂的运动学及接触约束，需要满足这些约束才能生成可行的操控轨迹。我们首先提出一种基于非线性规划的开环控制器设计，以符合这些约束条件。为设计闭环控制器，我们提出一种利用触觉传感器的物体与工具姿态估计方法。基于该触觉估计器，我们设计了基于模型预测控制的闭环控制器。通过使用六自由度机械臂在多种物体和工具上的操作任务，验证了所提算法的有效性。实验证明，我们的闭环控制器能在多种意外接触情况下成功执行工具操控任务。总结本文内容及硬件实验的视频见链接：https://youtu.be/VsClK04qDhk。