Robotic systems for manipulation in millimeter scale often use a camera with high magnification for visual feedback of the target region. However, the limited field-of-view (FoV) of the microscopic camera necessitates camera motion to capture a broader workspace environment. In this work, we propose an autonomous robotic control method to constrain a robot-held camera within a designated FoV. Furthermore, we model the camera extrinsics as part of the kinematic model and use camera measurements coupled with a U-Net based tool tracking to adapt the complete robotic model during task execution. As a proof-of-concept demonstration, the proposed framework was evaluated in a bi-manual setup, where the microscopic camera was controlled to view a tool moving in a pre-defined trajectory. The proposed method allowed the camera to stay 99.5% of the time within the real FoV, compared to 48.1% without the proposed adaptive control.

翻译：毫米尺度操作的机器人系统通常使用高倍率相机获取目标区域的视觉反馈。然而，显微相机的有限视场(FoV)迫使相机移动以捕获更广阔的工作空间环境。本文提出一种自主机器人控制方法，将机器人持相机约束在指定视场内。此外，我们将相机外参建模为运动学模型的一部分，并利用相机测量值与基于U-Net的工具跟踪相结合，在任务执行过程中自适应修正完整机器人模型。作为概念验证，该框架在双臂平台上进行了评估，其中显微相机被控制以观察沿预设轨迹运动的工具。所提方法使相机在真实视场内的停留时间达到99.5%，而未经自适应控制的对比组仅为48.1%。