Transformer Neural Networks are driving an explosion of activity and discovery in the field of Large Language Models (LLMs). In contrast, there have been only a few attempts to apply Transformers in engineering physics. Aiming to offer an easy entry point to physics-centric Transformers, we introduce a physics-informed Transformer model for solving the heat conduction problem in a 2D plate with Dirichlet boundary conditions. The model is implemented in the machine learning framework MLX and leverages the unified memory of Apple M-series processors. The use of MLX means that the models can be trained and perform predictions efficiently on personal machines with only modest memory requirements. To train, validate and test the Transformer model we solve the 2D heat conduction problem using central finite differences. Each finite difference solution in these sets is initialized with four random Dirichlet boundary conditions, a uniform but random internal temperature distribution and a randomly selected thermal diffusivity. Validation is performed in-line during training to monitor against over-fitting. The excellent performance of the trained model is demonstrated by predicting the evolution of the temperature field to steady state for the unseen test set of conditions.

翻译：Transformer神经网络正在推动大语言模型（LLM）领域的研究活动与发现呈现爆炸式增长。相比之下，将Transformer应用于工程物理学领域的尝试却寥寥无几。为了提供一个易于入门的、以物理学为核心的Transformer模型，我们引入了一种物理信息嵌入的Transformer模型，用于求解具有狄利克雷边界条件的二维平板热传导问题。该模型在机器学习框架MLX中实现，并利用了苹果M系列处理器的统一内存架构。使用MLX意味着该模型可以在个人计算机上高效地进行训练和预测，且仅需适中的内存要求。为了训练、验证和测试该Transformer模型，我们使用中心有限差分法求解了二维热传导问题。在这些数据集中，每个有限差分解均由四个随机的狄利克雷边界条件、一个均匀但随机的内部温度分布以及一个随机选择的热扩散系数初始化。在训练过程中进行在线验证，以监控过拟合现象。训练后模型的优异性能通过预测未见测试条件集下温度场向稳态演化的过程得到了验证。