Current unsupervised learning methods depend on end-to-end training via deep learning techniques such as self-supervised learning, with high computational requirements, or employ layer-by-layer training using bio-inspired approaches like Hebbian learning, using local learning rules incompatible with supervised learning. Both approaches are problematic for edge AI hardware that relies on sparse computational resources and would strongly benefit from alternating between unsupervised and supervised learning phases - thus leveraging widely available unlabeled data from the environment as well as labeled training datasets. To solve this challenge, in this work, we introduce a 'self-defined target' that uses Winner-Take-All (WTA) selectivity at the network's final layer, complemented by regularization through biologically inspired homeostasis mechanism. This approach, framework-agnostic and compatible with both global (Backpropagation) and local (Equilibrium propagation) learning rules, achieves a 97.6% test accuracy on the MNIST dataset. Furthermore, we demonstrate that incorporating a hidden layer enhances classification accuracy and the quality of learned features across all training methods, showcasing the advantages of end-to-end unsupervised training. Extending to semi-supervised learning, our method dynamically adjusts the target according to data availability, reaching a 96.6% accuracy with just 600 labeled MNIST samples. This result highlights our 'unsupervised target' strategy's efficacy and flexibility in scenarios ranging from abundant to no labeled data availability.
翻译:当前无监督学习方法要么依赖自监督学习等深度学习技术进行端到端训练,计算需求较高,要么采用赫布学习等类脑方法进行逐层训练,但所使用的局部学习规则与监督学习不兼容。对于依赖稀疏计算资源、且需在无监督与监督学习阶段间灵活切换——从而同时利用环境中广泛存在的未标注数据和标注训练数据集——的边缘AI硬件而言,这两种方法均存在明显缺陷。为解决这一挑战,本文提出一种"自定义目标"方法:在网络最终层采用胜者全取(WTA)选择性机制,并通过生物启发的稳态调节进行正则化修正。该方法与全局学习规则(反向传播)和局部学习规则(平衡传播)均兼容,且不受框架限制,在MNIST数据集上实现了97.6%的测试准确率。进一步研究表明,引入隐藏层能提升所有训练方法的分类准确率与学习特征质量,充分展现端到端无监督训练的优势。将方法扩展至半监督学习场景后,我们的方法可根据数据可获取性动态调整目标,仅用600个MNIST标注样本即可达到96.6%的准确率。这一结果凸显了"无监督目标"策略在从数据富集到完全无标注数据等多种场景下的高效性与灵活性。