This is a theoretical paper, as a companion paper of the keynote talk at the same conference AIEE 2023. In contrast to conscious learning, many projects in AI have employed so-called "deep learning" many of which seemed to give impressive performance. This paper explains that such performance data are deceptively inflated due to two misconducts: "data deletion" and "test on training set". This paper clarifies "data deletion" and "test on training set" in deep learning and why they are misconducts. A simple classification method is defined, called Nearest Neighbor With Threshold (NNWT). A theorem is established that the NNWT method reaches a zero error on any validation set and any test set using the two misconducts, as long as the test set is in the possession of the author and both the amount of storage space and the time of training are finite but unbounded like with many deep learning methods. However, many deep learning methods, like the NNWT method, are all not generalizable since they have never been tested by a true test set. Why? The so-called "test set" was used in the Post-Selection step of the training stage. The evidence that misconducts actually took place in many deep learning projects is beyond the scope of this paper.
翻译:这是一篇理论性论文,作为同一会议 AIEE 2023 主题演讲的配套论文。与有意识学习不同,人工智能领域的许多项目采用了所谓的“深度学习”,其中许多项目似乎取得了令人瞩目的性能。本文解释了此类性能数据因两种不当行为而被虚假夸大:“数据删除”和“在训练集上测试”。本文澄清了深度学习中的“数据删除”和“在训练集上测试”以及它们为何是不当行为。定义了一种简单的分类方法,称为带阈值的最近邻(NNWT)。建立了一个定理:只要测试集由作者持有,且存储空间和训练时间像许多深度学习方法一样有限但无界,NNWT 方法通过使用这两种不当行为,可在任何验证集和任何测试集上达到零错误。然而,许多深度学习方法(如 NNWT 方法)均不可泛化,因为它们从未经过真正测试集的检验。原因何在?所谓“测试集”被用于训练阶段的后选择步骤。关于这些不当行为是否实际发生于众多深度学习项目中的证据,不在本文讨论范围之内。