With Artificial Intelligence systems increasingly applied in consequential domains, researchers have begun to ask how these systems ought to act in ethically charged situations where even humans lack consensus. In the Moral Machine project, researchers crowdsourced answers to "Trolley Problems" concerning autonomous vehicles. Subsequently, Noothigattu et al. (2018) proposed inferring linear functions that approximate each individual's preferences and aggregating these linear models by averaging parameters across the population. In this paper, we examine this averaging mechanism, focusing on fairness concerns in the presence of strategic effects. We investigate a simple setting where the population consists of two groups, with the minority constituting an {\alpha} < 0.5 share of the population. To simplify the analysis, we consider the extreme case in which within-group preferences are homogeneous. Focusing on the fraction of contested cases where the minority group prevails, we make the following observations: (a) even when all parties report their preferences truthfully, the fraction of disputes where the minority prevails is less than proportionate in {\alpha}; (b) the degree of sub-proportionality grows more severe as the level of disagreement between the groups increases; (c) when parties report preferences strategically, pure strategy equilibria do not always exist; and (d) whenever a pure strategy equilibrium exists, the majority group prevails 100% of the time. These findings raise concerns about stability and fairness of preference vector averaging as a mechanism for aggregating diverging voices. Finally, we discuss alternatives, including randomized dictatorship and median-based mechanisms.

翻译：随着人工智能系统越来越多地应用于重要领域，研究者开始探讨：在人类自身也缺乏共识的道德困境情境中，这些系统应如何行动。在“道德机器”项目中，研究者通过众包方式收集了关于自动驾驶汽车“电车难题”的答案。随后，Noothigattu等人（2018）提出推断线性函数以近似个体偏好，并通过平均群体参数来聚合这些线性模型。本文聚焦于这一平均机制，关注存在策略效应时的公平性问题。我们考察了一个简单场景：群体由两个子群组成，少数群体占人口比例{\alpha} < 0.5。为简化分析，我们假设组内偏好同质。通过分析少数群体在争议案件中获胜的比例，我们得出以下发现：(a) 即使所有个体真实报告偏好，少数群体在争议中获胜的比例也低于{\alpha}的等比份额；(b) 子比例偏离程度随群体间分歧加剧而增大；(c) 当个体策略性报告偏好时，纯策略均衡并非总是存在；(d) 一旦纯策略均衡存在，多数群体将获得100%的胜率。这些发现对偏好向量平均作为聚合分歧声音的机制提出了稳定性和公平性方面的质疑。最后，我们讨论了包括随机独裁和中位数机制在内的替代方案。