In our work we not explicitly hint that it is a misconception to think that humans learn fast. Learning process takes time. Babies start learning to move in the restricted liquid area called placenta. Children often are limited by underdeveloped body. Even adults are not allowed to participate in complex competitions right away. However, with robots, when learning from scratch, we often don't have the privilege of waiting for dozen millions of steps. "Swaddling" regularization is responsible for restraining an agent in rapid but unstable development penalizing action strength in a specific way not affecting actions directly. The Symphony, Transitional-policy Deterministic Actor and Critic algorithm, is a concise combination of different ideas for possibility of training humanoid robots from scratch with Sample Efficiency, Sample Proximity and Safety of Actions in mind. It is no secret that continuous increase in Gaussian noise without appropriate smoothing is harmful for motors and gearboxes. Compared to Stochastic algorithms, we set a limited parametric noise and promote a reduced strength of actions, safely increasing entropy, since the actions are kind of immersed in weaker noise. When actions require more extreme values, actions rise above the weak noise. Training becomes empirically much safer for both the environment around and the robot's mechanisms. We use Fading Replay Buffer: using a fixed formula containing the hyperbolic tangent, we adjust the batch sampling probability: the memory contains a recent memory and a long-term memory trail. Fading Replay Buffer allows us to use Temporal Advantage when we improve the current Critic Network prediction compared to the exponential moving average. Temporal Advantage allows us to update Actor and Critic in one pass, as well as combine Actor and Critic in one Object and implement their Losses in one line.

翻译：在我们的工作中，我们并未明确暗示“人类学习速度快”是一种误解。学习过程需要时间。婴儿始于在称为胎盘的受限液体区域内学习运动。儿童常受限于未发育完全的身体。即使是成年人，也无法立即参与复杂的竞技活动。然而，对于机器人而言，当从零开始学习时，我们通常没有等待数千万步的奢侈条件。“襁褓”正则化旨在约束智能体在快速但不稳定的发展过程中，以一种不影响动作直接执行的特殊方式惩罚动作强度。Symphony，即过渡策略确定性行动者-评论者算法，是多种理念的简洁结合，旨在实现从零开始训练仿人机器人，并兼顾样本效率、样本邻近性与动作安全性。众所周知，未经适当平滑的高斯噪声持续增加对电机和齿轮箱有害。与随机算法相比，我们设置了有限的参数化噪声，并促进降低动作强度，从而安全地增加熵，因为动作仿佛浸没在较弱的噪声中。当动作需要更极端的值时，动作会从弱噪声中凸显出来。经验表明，这种训练方式对周围环境及机器人机械结构都更为安全。我们使用衰减回放缓冲区：通过一个包含双曲正切函数的固定公式，我们调整批次采样概率：记忆体中包含近期记忆与长期记忆轨迹。衰减回放缓冲区使我们能够在改进当前评论者网络预测（相较于指数移动平均）时利用时序优势。时序优势允许我们在一次前向传播中更新行动者与评论者，并将行动者与评论者结合于同一对象中，用一行代码实现两者的损失函数。