This study investigates the effectiveness of synthetic data for sim-to-real transfer in object detection under constrained data conditions and embedded deployment requirements. Synthetic datasets were generated in NVIDIA Isaac Sim and combined with limited real-world fruit images to train YOLO-based detection models under real-only, synthetic-only, and hybrid regimes. Performance was evaluated on two test datasets: an in-domain dataset with conditions matching the training data and a domain shift dataset containing real fruit and different background conditions. Results show that models trained exclusively on real data achieve the highest accuracy, while synthetic-only models exhibit reduced performance due to a domain gap. Hybrid training strategies significantly improve performance compared to synthetic-only approaches and achieve results close to real-only training while reducing the need for manual annotation. Under domain shift conditions, all models show performance degradation, with hybrid models providing improved robustness. The trained models were successfully deployed on a Jetson Orin NX using TensorRT optimization, achieving real-time inference performance. The findings highlight that synthetic data is most effective when used in combination with real data and that deployment constraints must be considered alongside detection accuracy.

翻译：本研究探究了在数据受限条件和嵌入式部署需求下，合成数据在目标检测虚实迁移中的有效性。我们在NVIDIA Isaac Sim中生成合成数据集，并将其与有限的真实水果图像相结合，在纯真实、纯合成及混合三种训练模式下训练基于YOLO的检测模型。模型性能在两个测试数据集上进行了评估：一个与训练数据条件匹配的域内数据集，以及一个包含真实水果与不同背景条件的域迁移数据集。结果表明，仅使用真实数据训练的模型准确率最高，而纯合成数据训练的模型因域差异性能下降。相比纯合成方案，混合训练策略能显著提升性能，其检测结果接近纯真实训练，同时减少了对人工标注的需求。在域迁移条件下，所有模型性能均有下降，但混合模型展现出更强的鲁棒性。通过TensorRT优化，训练模型成功部署于Jetson Orin NX平台，实现了实时推理性能。研究结论强调：合成数据在与真实数据结合使用时效果最佳，且部署约束需与检测精度同时纳入考量。