Design of process control scheme is critical for quality assurance to reduce variations in manufacturing systems. Taking semiconductor manufacturing as an example, extensive literature focuses on control optimization based on certain process models (usually linear models), which are obtained by experiments before a manufacturing process starts. However, in real applications, pre-defined models may not be accurate, especially for a complex manufacturing system. To tackle model inaccuracy, we propose a model-free reinforcement learning (MFRL) approach to conduct experiments and optimize control simultaneously according to real-time data. Specifically, we design a novel MFRL control scheme by updating the distribution of disturbances using Bayesian inference to reduce their large variations during manufacturing processes. As a result, the proposed MFRL controller is demonstrated to perform well in a nonlinear chemical mechanical planarization (CMP) process when the process model is unknown. Theoretical properties are also guaranteed when disturbances are additive. The numerical studies also demonstrate the effectiveness and efficiency of our methodology.

翻译：过程控制方案的设计对于减少制造系统变异、保障产品质量至关重要。以半导体制造为例，现有研究多基于实验预先获得的特定过程模型（通常为线性模型）进行控制优化。然而在实际应用中，预先定义的模型可能不够精确，尤其是在复杂制造系统中。针对模型不准确问题，我们提出一种无模型强化学习（MFRL）方法，可同时根据实时数据进行实验探索与控制优化。具体而言，我们设计了一种新型MFRL控制方案，通过贝叶斯推断更新扰动分布，以降低制造过程中大幅变异的影响。实验结果表明，所提出的MFRL控制器在过程模型未知的非线性化学机械平坦化（CMP）工艺中表现优异。当扰动为加性时，其理论性质亦可得到保障。数值研究进一步验证了该方法的高效性与有效性。