We study the problem of trajectory optimization in settings where the system dynamics are unknown and the controller cannot simulate candidate trajectories through a model at test time. When an offline dataset of trajectories is available, an agent could directly learn a trajectory generator by distribution matching. However, this approach only recovers the behavior distribution in the dataset, and does not in general produce a model that minimizes a desired cost criterion. In this work, we propose Drifting Receding Horizon Control (RHC), an offline trajectory optimization framework that combines drifting generative models with receding-horizon planning under unknown dynamics. The goal of Drifting RHC is to learn, from an offline dataset of trajectories, a conditional distribution over trajectories that is both supported by the data and biased toward optimal plans. We show that the target distribution of Drifting RHC is the unique solution of an objective that trades off optimality with closeness to the offline prior. We further bound the gap between the predicted and the executed cost of a generated plan by its consistency with the dynamics, and introduce a rollout-consistency loss that distills a learned model into the generator at training time only. On a mass-spring-damper, a cart-pole and a planar quadrotor, Drifting RHC attains the lowest regret among generative and model-based planners, outperforms DeePC on the nonlinear systems, and plans $9$ to $83$ times faster than iLQR, model-based and diffusion-based planners.

翻译：暂无翻译