The replica-exchange Monte-Carlo (RXMC) method is a powerful Markov-chain Monte-Carlo algorithm for sampling from multi-modal distributions, which are challenging for conventional methods. The sampling efficiency of the RXMC method depends highly on the selection of the temperatures, and finding optimal temperatures remains a challenge. In this study, we propose a refined online temperature selection method by extending the gradient-based optimization framework proposed previously. Building upon the existing temperature update approach, we introduce a reparameterization technique to strictly enforce physical constraints, such as the monotonic ordering of inverse temperatures, which were not explicitly addressed in the original formulation. The proposed method defines the variance of acceptance rates between adjacent replicas as a loss function, estimates its gradient using differential information from the sampling process, and optimizes the temperatures via gradient descent. We demonstrate the effectiveness of our method through experiments on benchmark spin systems, including the two-dimensional ferromagnetic Ising model, the two-dimensional ferromagnetic XY model, and the three-dimensional Edwards-Anderson model. Our results show that the method successfully achieves uniform acceptance rates and reduces round-trip times across the temperature space. Furthermore, our proposed method offers a significant advantage over recently proposed policy gradient method that require careful hyperparameter tuning, while simultaneously preventing the constraint violations that destabilize optimization.

翻译：副本交换蒙特卡洛（RXMC）方法是一种强大的马尔可夫链蒙特卡洛算法，适用于从多模态分布中采样，这类分布对传统方法具有挑战性。RXMC方法的采样效率高度依赖于温度的选择，而寻找最优温度仍是一个难题。在本研究中，我们通过扩展先前提出的基于梯度的优化框架，提出了一种改进的在线温度选择方法。在现有温度更新方法的基础上，我们引入了一种重参数化技术，以严格强制执行物理约束（如逆温度的单调排序），这些约束在原始公式中未得到明确处理。该方法将相邻副本间接受率的方差定义为损失函数，利用采样过程中的微分信息估计其梯度，并通过梯度下降优化温度。我们通过在基准自旋系统上的实验证明了该方法的有效性，包括二维铁磁伊辛模型、二维铁磁XY模型和三维爱德华兹-安德森模型。实验结果表明，该方法成功实现了均匀的接受率，并减少了跨越温度空间的往返时间。此外，与最近提出的需要仔细超参数调整的策略梯度方法相比，我们提出的方法具有显著优势，同时避免了破坏优化稳定性的约束违反问题。