In this paper, we study Mahalanobis-guided latent out-of-distribution (OOD) detection for test-time RL controller switching in nonlinear time-varying systems. RL controllers can quickly control high-dimensional systems within the training distribution, but their performance can degrade when time-varying dynamics produce unseen observations. We consider a combined ES--DRL controller, where RL provides fast in-distribution actions and bounded extremum seeking (ES) provides robust model-independent control under OOD operation. The key challenge is deciding when to switch. We train a variational autoencoder (VAE) on in-distribution beam-profile observations and use Mahalanobis distance in the VAE latent space to detect OOD beam profiles at test time. This OOD decision sets a binary switch that selects either the RL controller or the ES controller. We evaluate the approach in safety-critical particle accelerator control. In this setting, spatial magnet motion creates OOD beam profiles that were not seen during RL training. Visualization of the VAE latent space shows that the proposed method identifies this OOD scenario and provides an interpretable signal for switching between RL and ES in the combined controller.

翻译：本文研究了非线性时变系统中基于马氏距离引导的潜在异常分布检测，用于测试时强化学习控制器切换。RL控制器能在训练分布内快速控制高维系统，但当时变动力学产生未见过的观测时，其性能可能下降。我们考虑联合的ES-DRL控制器架构，其中RL提供分布内快速动作，而有界极值搜索方法在异常分布运行下提供鲁棒的模型无关控制。关键挑战在于切换时机的决策。我们使用分布内束流剖面观测训练变分自编码器，并在VAE潜在空间中以马氏距离检测测试时的异常束流剖面。该异常决策生成二值切换信号，选择RL控制器或ES控制器。我们在安全关键的粒子加速器控制场景中评估该方法。在此场景中，空间磁体运动产生的异常束流剖面未出现在RL训练数据中。VAE潜在空间可视化表明，所提方法能够识别此类异常场景，并为联合控制器中RL与ES之间的切换提供可解释信号。