This paper presents the concepts of Artificial Intelligence, Multi-Agent-Systems, Coordination, Intelligent Robotics and Deep Reinforcement Learning. Emphasis is given on and how AI and DRL, may be efficiently used to create efficient robot skills and coordinated robotic teams, capable of performing very complex actions and tasks, such as playing a game of soccer. The paper also presents the concept of robotic soccer and the vision and structure of the RoboCup initiative with emphasis on the Humanoid Simulation 3D league and the new challenges this competition, poses. The final topics presented at the paper are based on the research developed/coordinated by the author throughout the last 22 years in the context of the FCPortugal project. The paper presents a short description of the coordination methodologies developed, such as: Strategy, Tactics, Formations, Setplays, and Coaching Languages and the use of Machine Learning to optimize the use of this concepts. The topics presented also include novel stochastic search algorithms for black box optimization and their use in the optimization of omnidirectional walking skills, robotic multi-agent learning and the creation of a humanoid kick with controlled distance. Finally, new applications using variations of the Proximal Policy Optimization algorithm and advanced modelling for robot and multi-robot learning are briefly explained with emphasis for our new humanoid sprinting and running skills and an amazing humanoid robot soccer dribbling skill. FCPortugal project enabled us to publish more than 100 papers and win several competitions in different leagues and many scientific awards at RoboCup. In total, our team won more than 40 awards in international competitions including a clear victory at the Simulation 3D League at RoboCup 2022 competition, scoring 84 goals and conceding only 2.

翻译：本文阐述了人工智能、多智能体系统、协调、智能机器人与深度强化学习等概念，重点探讨了人工智能与深度强化学习如何被高效利用，以创建具备强大能力的机器人技能和协调型机器人团队，使其能够执行诸如足球比赛等高度复杂的动作与任务。同时，本文还介绍了机器人足球的理念，以及RoboCup计划的愿景与结构，特别关注人形仿真3D联赛及其带来的新挑战。论文最后部分基于作者过去22年间在FCPortugal项目中开展/协调的研究工作，简要描述了所开发的协调方法，包括：策略、战术、阵型、固定战术套路及教练语言，并阐述了如何利用机器学习优化这些概念的应用。此外，文中还介绍了用于黑箱优化的新型随机搜索算法，及其在全方位行走技能优化、机器人多智能体学习以及创建具有可控距离的人形踢球动作中的应用。最后，本文简要解释了基于近端策略优化算法变体的新应用，以及用于机器人与多机器人学习的高级建模，重点介绍了我们新开发的人形短跑与奔跑技能，以及一项令人惊叹的人形机器人足球控球技能。FCPortugal项目使我们能够发表超过100篇论文，并在RoboCup不同联赛中赢得多项比赛及众多科学奖项。总计，我们的团队在国际竞赛中赢得了超过40个奖项，包括在2022年RoboCup仿真3D联赛中以进84球仅失2球的成绩取得完胜。