Recently, there has been an increasing interest in exploring the application of multiobjective optimization (MOO) in machine learning (ML). The interest is driven by the numerous situations in real-life applications where multiple objectives need to be optimized simultaneously. A key aspect of MOO is the existence of a Pareto set, rather than a single optimal solution, which illustrates the inherent trade-offs between objectives. Despite its potential, there is a noticeable lack of satisfactory literature that could serve as an entry-level guide for ML practitioners who want to use MOO. Hence, our goal in this paper is to produce such a resource. We critically review previous studies, particularly those involving MOO in deep learning (using Physics-Informed Neural Networks (PINNs) as a guiding example), and identify misconceptions that highlight the need for a better grasp of MOO principles in ML. Using MOO of PINNs as a case study, we demonstrate the interplay between the data loss and the physics loss terms. We highlight the most common pitfalls one should avoid while using MOO techniques in ML. We begin by establishing the groundwork for MOO, focusing on well-known approaches such as the weighted sum (WS) method, alongside more complex techniques like the multiobjective gradient descent algorithm (MGDA). Additionally, we compare the results obtained from the WS and MGDA with one of the most common evolutionary algorithms, NSGA-II. We emphasize the importance of understanding the specific problem, the objective space, and the selected MOO method, while also noting that neglecting factors such as convergence can result in inaccurate outcomes and, consequently, a non-optimal solution. Our goal is to offer a clear and practical guide for ML practitioners to effectively apply MOO, particularly in the context of DL.

翻译：近年来，多目标优化（MOO）在机器学习（ML）领域的应用日益受到关注。这一趋势源于现实应用中需要同时优化多个目标的众多场景。多目标优化的关键在于存在帕累托集而非单一最优解，这揭示了目标间固有的权衡关系。尽管潜力巨大，但目前仍缺乏能够作为ML从业者入门指南的满意文献。因此，本文旨在填补这一空白。我们批判性回顾了以往研究，特别是涉及深度学习中的MOO（以物理信息神经网络（PINNs）为引导案例），并识别出凸显ML从业者亟需深入理解MOO原理的认知误区。通过以PINNs的MOO为案例，我们展示了数据损失项与物理损失项之间的相互作用，并重点阐述了在ML中使用MOO技术时应避免的常见陷阱。我们首先建立MOO基础框架，聚焦于加权求和法（WS）等经典方法，以及多目标梯度下降算法（MGDA）等更复杂的技术。此外，我们还将WS和MGDA的结果与最常用的进化算法之一NSGA-II进行了比较。我们强调理解具体问题、目标空间及所选MOO方法的重要性，同时指出忽略收敛性等因素可能导致不准确结果及非最优方案。本文旨在为ML从业者提供清晰实用的指导，助力其有效应用MOO，特别是在深度学习背景下。