As the reliability of the robot's perception correlates with the number of integrated sensing modalities to tackle uncertainty, a practical solution to manage these sensors from different computers, operate them simultaneously, and maintain their real-time performance on the existing robotic system with minimal effort is needed. In this work, we present an end-to-end software-hardware framework, namely ExtPerFC, that supports both conventional hardware and software components and integrates machine learning object detectors without requiring an additional dedicated graphic processor unit (GPU). We first design our framework to achieve real-time performance on the existing robotic system, guarantee configuration optimization, and concentrate on code reusability. We then mathematically model and utilize our transfer learning strategies for 2D object detection and fuse them into depth images for 3D depth estimation. Lastly, we systematically test the proposed framework on the Baxter robot with two 7-DOF arms, a four-wheel mobility base, and an Intel RealSense D435i RGB-D camera. The results show that the robot achieves real-time performance while executing other tasks (e.g., map building, localization, navigation, object detection, arm moving, and grasping) simultaneously with available hardware like Intel onboard CPUS/GPUs on distributed computers. Also, to comprehensively control, program, and monitor the robot system, we design and introduce an end-user application. The source code is available at https://github.com/tuantdang/perception_framework.

翻译：摘要：由于机器人感知的可靠性与其为应对不确定性而集成的感知模态数量相关，因此亟需一种实用方案，能够管理来自不同计算机的传感器、使其同步运行，并在现有机器人系统上以最小代价维持其实时性能。本文提出一种端到端的软硬件框架ExtPerFC，其不仅支持传统硬件与软件组件，还能集成机器学习目标检测器，且无需额外专用图形处理器单元。我们首先设计该框架以实现现有机器人系统的实时性能，确保配置优化，并聚焦代码可复用性。随后，我们通过数学建模并应用迁移学习策略实现二维目标检测，将其融合至深度图像以完成三维深度估计。最后，我们在搭载两条7自由度机械臂、四轮移动底盘及Intel RealSense D435i RGB-D相机的Baxter机器人上系统性地测试了所提框架。结果表明，在分布式计算机上利用Intel板载CPU/GPU等现有硬件，机器人可在执行地图构建、定位、导航、目标检测、机械臂运动与抓取等其他任务的同时实现实时性能。此外，为全面控制、编程与监控机器人系统，我们设计并引入了一款终端用户应用程序。源代码发布于https://github.com/tuantdang/perception_framework。