Training machine and deep learning models directly on extremely resource-constrained devices is the next challenge in the field of tiny machine learning. The related literature in this field is very limited, since most of the solutions focus only on on-device inference or model adaptation through online learning, leaving the training to be carried out on external Cloud services. An interesting technological perspective is to exploit Federated Learning (FL), which allows multiple devices to collaboratively train a shared model in a distributed way. However, the main drawback of state-of-the-art FL algorithms is that they are not suitable for running on tiny devices. For the first time in the literature, in this paper we introduce TIFeD, a Tiny Integer-based Federated learning algorithm with Direct Feedback Alignment (DFA) entirely implemented by using an integer-only arithmetic and being specifically designed to operate on devices with limited resources in terms of memory, computation and energy. Besides the traditional full-network operating modality, in which each device of the FL setting trains the entire neural network on its own local data, we propose an innovative single-layer TIFeD implementation, which enables each device to train only a portion of the neural network model and opens the door to a new way of distributing the learning procedure across multiple devices. The experimental results show the feasibility and effectiveness of the proposed solution. The proposed TIFeD algorithm, with its full-network and single-layer implementations, is made available to the scientific community as a public repository.

翻译：在资源极度受限的设备上直接训练机器学习和深度学习模型，是微型机器学习领域的下一个挑战。该领域的相关文献非常有限，因为大多数解决方案仅关注设备端推理或通过在线学习进行模型适配，而将训练任务交由外部云服务完成。一个有趣的技术视角是利用联邦学习（FL），它允许多个设备以分布式方式协同训练一个共享模型。然而，现有先进联邦学习算法的主要缺点在于它们不适合在微型设备上运行。本文首次在文献中介绍了TIFeD——一种基于整数的微型联邦学习算法，采用直接反馈对齐（DFA）机制，完全使用纯整数运算实现，并专门设计用于在内存、计算和能源资源有限的设备上运行。除了传统的全网络操作模式（即联邦学习设置中的每个设备在其本地数据上训练整个神经网络）之外，我们提出了一种创新的单层TIFeD实现方案，使得每个设备仅训练神经网络模型的一部分，这为跨多个设备分布式学习过程开辟了新的途径。实验结果表明了所提方案的可行性和有效性。所提出的TIFeD算法及其全网络和单层实现已作为公共资源库向科学界开放。