Robotic systems for manipulation in millimeter scale often use a camera with high magnification for visual feedback of the target region. However, the limited field-of-view (FoV) of the microscopic camera necessitates camera motion to capture a broader workspace environment. In this work, we propose an autonomous robotic control method to constrain a robot-held camera within a designated FoV. Furthermore, we model the camera extrinsics as part of the kinematic model and use camera measurements coupled with a U-Net based tool tracking to adapt the complete robotic model during task execution. As a proof-of-concept demonstration, the proposed framework was evaluated in a bi-manual setup, where the microscopic camera was controlled to view a tool moving in a pre-defined trajectory. The proposed method allowed the camera to stay 94.1% of the time within the real FoV, compared to 54.4% without the proposed adaptive control.

翻译：毫米级操作机器人系统常采用高倍率显微相机对目标区域进行视觉反馈。然而，显微相机的有限视场（FoV）要求相机运动以覆盖更广的操作环境。本文提出一种自主机器人控制方法，将机器人持相机约束在指定视场内。进一步，我们将相机外参建模为运动学模型的一部分，并结合基于U-Net的工具跟踪测量值，在任务执行过程中自适应调整完整机器人模型。作为概念验证，该框架在双臂设置中进行了评估：控制显微相机观察沿预定轨迹运动的工具。实验结果表明，所提方法使相机在真实视场内的停留时间达94.1%，而未使用自适应控制时仅为54.4%。