Due to the point cloud's irregular and unordered geometry structure, conventional knowledge distillation technology lost a lot of information when directly used on point cloud tasks. In this paper, we propose Feature Adversarial Distillation (FAD) method, a generic adversarial loss function in point cloud distillation, to reduce loss during knowledge transfer. In the feature extraction stage, the features extracted by the teacher are used as the discriminator, and the students continuously generate new features in the training stage. The feature of the student is obtained by attacking the feedback from the teacher and getting a score to judge whether the student has learned the knowledge well or not. In experiments on standard point cloud classification on ModelNet40 and ScanObjectNN datasets, our method reduced the information loss of knowledge transfer in distillation in 40x model compression while maintaining competitive performance.

翻译：由于点云具有不规则且无序的几何结构，传统知识蒸馏技术在直接应用于点云任务时会损失大量信息。本文提出了一种点云蒸馏中的通用对抗损失函数——特征对抗蒸馏（FAD）方法，以减少知识迁移过程中的信息损失。在特征提取阶段，教师模型提取的特征被用作判别器，学生模型在训练阶段持续生成新特征。学生模型的特征通过攻击教师模型的反馈获得，并依据得分判断学生是否充分掌握所学知识。在ModelNet40和ScanObjectNN数据集上的标准点云分类实验中，我们的方法在实现40倍模型压缩的同时，显著降低了蒸馏过程中知识迁移的信息损失，并保持了具有竞争力的性能。