The growing IoT landscape requires effective server deployment strategies to meet demands including real-time processing and energy efficiency. This is complicated by heterogeneous, dynamic applications and servers. To address these challenges, we propose ReinFog, a modular distributed software empowered with Deep Reinforcement Learning (DRL) for adaptive resource management across edge/fog and cloud environments. ReinFog enables the practical development/deployment of various centralized and distributed DRL techniques for resource management in edge/fog and cloud computing environments. It also supports integrating native and library-based DRL techniques for diverse IoT application scheduling objectives. Additionally, ReinFog allows for customizing deployment configurations for different DRL techniques, including the number and placement of DRL Learners and DRL Workers in large-scale distributed systems. Besides, we propose a novel Memetic Algorithm for DRL Component (e.g., DRL Learners and DRL Workers) Placement in ReinFog named MADCP, which combines the strengths of Genetic Algorithm, Firefly Algorithm, and Particle Swarm Optimization. Experiments reveal that the DRL mechanisms developed within ReinFog have significantly enhanced both centralized and distributed DRL techniques implementation. These advancements have resulted in notable improvements in IoT application performance, reducing response time by 45%, energy consumption by 39%, and weighted cost by 37%, while maintaining minimal scheduling overhead. Additionally, ReinFog exhibits remarkable scalability, with a rise in DRL Workers from 1 to 30 causing only a 0.3-second increase in startup time and around 2 MB more RAM per Worker. The proposed MADCP for DRL component placement further accelerates the convergence rate of DRL techniques by up to 38%.

翻译：日益增长的物联网（IoT）场景需要有效的服务器部署策略，以满足实时处理和能效等需求。异构且动态的应用与服务器使这一问题更加复杂。为应对这些挑战，我们提出了ReinFog，这是一个模块化的分布式软件框架，通过深度强化学习（DRL）赋能，实现跨边缘/雾与云环境的自适应资源管理。ReinFog支持在边缘/雾与云计算环境中，针对资源管理任务实际开发和部署各种集中式与分布式DRL技术。它还支持集成基于原生及库的DRL技术，以实现多样化的物联网应用调度目标。此外，ReinFog允许为不同的DRL技术定制部署配置，包括在大规模分布式系统中DRL学习器与DRL工作器的数量与放置位置。除此之外，我们提出了一种新颖的用于ReinFog中DRL组件（例如DRL学习器与DRL工作器）放置的模因算法，称为MADCP，它结合了遗传算法、萤火虫算法和粒子群优化的优势。实验表明，在ReinFog中开发的DRL机制显著增强了集中式与分布式DRL技术的实现。这些改进带来了物联网应用性能的显著提升，响应时间降低了45%，能耗降低了39%，加权成本降低了37%，同时保持了极低的调度开销。此外，ReinFog展现出卓越的可扩展性，当DRL工作器数量从1增加到30时，启动时间仅增加0.3秒，且每个工作器仅增加约2 MB的RAM占用。所提出的用于DRL组件放置的MADCP进一步将DRL技术的收敛速度提升了高达38%。