This article introduces a framework that is meant to assist in mitigating societal risks that software can pose. Concretely, this encompasses facets of software doping as well as unfairness and discrimination in high-risk decision-making systems. The term software doping refers to software that contains surreptitiously added functionality that is against the interest of the user. A prominent example of software doping are the tampered emission cleaning systems that were found in millions of cars around the world when the diesel emissions scandal surfaced. The first part of this article combines the formal foundations of software doping analysis with established probabilistic falsification techniques to arrive at a black-box analysis technique for identifying undesired effects of software. We apply this technique to emission cleaning systems in diesel cars but also to high-risk systems that evaluate humans in a possibly unfair or discriminating way. We demonstrate how our approach can assist humans-in-the-loop to make better informed and more responsible decisions. This is to promote effective human oversight, which will be a central requirement enforced by the European Union's upcoming AI Act. We complement our technical contribution with a juridically, philosophically, and psychologically informed perspective on the potential problems caused by such systems.

翻译：本文介绍了一个旨在协助缓解软件可能带来的社会风险的框架。具体而言，这涵盖了软件掺杂以及高风险决策系统中的不公平与歧视现象。"软件掺杂"指软件中包含的、秘密添加且违背用户利益的额外功能。其中一个典型例证是柴油排放丑闻中曝光的全球数百万辆汽车安装的篡改式排放清洁系统。本文第一部分将软件掺杂分析的形式化基础与已有的概率性伪造技术相结合，形成一种用于识别软件意外效应的黑盒分析技术。我们将该技术应用于柴油车排放清洁系统，同时也用于评估可能不公平或歧视性地对待人类的高风险系统。我们展示了该方法如何辅助人在回路系统做出更明智、更负责任的决策，从而促进有效的人类监督——这将是欧盟即将出台的《人工智能法案》所强制执行的核心要求。除技术贡献外，我们还从法学、哲学及心理学视角补充了对这类系统潜在问题的分析。