Precise control and positioning of autonomous underwater vehicles (AUVs) is critical for sampling, maintenance, and survey applications. Reinforcement learning (RL) offers rapid controller development while handling a range of deployment parameters via domain randomization (DR). However, DR is limited by the underlying simulation's capacity to model real physics such as drag, which is a large contributor to sim-to-real gaps. Computational fluid dynamics (CFD) provides high-fidelity drag models but has significant computational cost. Thus, in this paper we train surrogate approximations of CFD data of a given vehicle, providing drag estimates 700,000x faster than solving for drag at each time step with CFD within the RL training pipeline. We deploy a Proximal Policy Optimization (PPO) policy zero-shot on a 6-DOF AUV in which policy training is performed on these surrogate drag models (SDMs), providing analysis on zero-shot transfer, reward shaping sensitivity, and physical parameter perturbations. On 15 identical segments in the ocean, we observe median gains of 19% in mean tracking error and 31% in control effort compared to the equivalent inertia box model. Our SDM based RL controller better predicts zero-shot transfer and reaches the most waypoints across all three reward shaping choices that we evaluate. Finally, we add 0.907 kg to the vehicle perturbing the mass, center-of-mass, and center-of-buoyancy. The policy reaches 100% of waypoints only when the SDM drag model is used; the policy fails to reach any of the 15 waypoints when the other drag models are used.

翻译：暂无翻译