To train well-performing generalizing neural networks, sufficiently large and diverse datasets are needed. Collecting data while adhering to privacy legislation becomes increasingly difficult and annotating these large datasets is both a resource-heavy and time-consuming task. An approach to overcome these difficulties is to use synthetic data since it is inherently scalable and can be automatically annotated. However, how training on synthetic data affects the layers of a neural network is still unclear. In this paper, we train the YOLOv3 object detector on real and synthetic images from city environments. We perform a similarity analysis using Centered Kernel Alignment (CKA) to explore the effects of training on synthetic data on a layer-wise basis. The analysis captures the architecture of the detector while showing both different and similar patterns between different models. With this similarity analysis we want to give insights on how training synthetic data affects each layer and to give a better understanding of the inner workings of complex neural networks. The results show that the largest similarity between a detector trained on real data and a detector trained on synthetic data was in the early layers, and the largest difference was in the head part. The results also show that no major difference in performance or similarity could be seen between frozen and unfrozen backbone.

翻译：为训练性能良好且具有泛化能力的神经网络，需要足够大且多样化的数据集。在遵守隐私法规的前提下收集数据日益困难，而标注这些大型数据集既耗费资源又耗时。克服这些困难的一种方法是使用合成数据，因为其天然可扩展且能自动标注。然而，关于在合成数据上训练如何影响神经网络各层仍然尚不明确。本文在来自城市环境的真实与合成图像上训练了YOLOv3目标检测器。我们采用中心核对齐（CKA）进行相似性分析，以逐层探究合成数据训练带来的影响。该分析捕捉了检测器的架构，同时展示了不同模型之间的差异模式与相似模式。借助这一相似性分析，我们希望揭示合成数据训练如何影响每一层，并帮助更深入地理解复杂神经网络的内部运作机制。结果表明，在真实数据与合成数据训练的检测器中，早期层的相似性最大，而头部部分的差异最大。结果还显示，冻结与未冻结主干网络之间在性能或相似性上未发现显著差异。