With the rapid rise of AI coding agents, the fundamental premise of what it means to be a software engineer is in question. In this vision paper, we examine what it means for an AI agent to be considered a software engineer and then critically think about what makes such an agent trustworthy. Grounded in established definitions of SE (SE) and informed by recent research on agentic AI systems, we conceptualise AI software engineers as participants in human-AI SE teams composed of human software engineers and AI agents, and we distinguish trustworthiness as a key property of these systems and actors rather than a subjective human attitude. Extending on historical perspectives and emerging visions, we identify key dimensions that contribute to the trustworthiness of AI software engineers, spanning technical quality, transparency and accountability, epistemic humility, and societal and ethical alignment. Beyond defining these dimensions, we address a critical but underexplored challenge: how trustworthiness can be operationalised in practice. We therefore introduce the notion of evidence-centric inspection, arguing that developers should evaluate selective signals and justifications of trustworthiness rather than raw outputs, and we outline implications for rethinking verification, validation, and code review in human-AI SE teams.

翻译：随着人工智能编码代理的快速崛起，软件工程师这一角色的根本前提正面临质疑。在这篇愿景论文中，我们探讨了人工智能代理被视为软件工程师的条件，并批判性地思考了使此类代理值得信赖的因素。基于软件工程的既有定义，并结合近年来关于自主人工智能系统的研究，我们将人工智能软件工程师概念化为由人类软件工程师与人工智能代理共同组成的人-人工智能软件工程团队中的参与者。我们强调将可信赖性视为这些系统与行为者的关键属性，而非主观的人类态度。在历史观点与新兴愿景的基础上，我们识别出构成人工智能软件工程师可信赖性的关键维度，涵盖技术质量、透明度与问责性、认知谦逊，以及社会与伦理契合。除了定义这些维度，我们还解决了一个关键但尚不充分探索的挑战：如何在实践中实现可信赖性。为此，我们引入基于证据的审查概念，主张开发者应评估关于可信赖性的选择性信号与论证，而非原始输出。最后，我们概述了这一概念对重新思考人-人工智能软件工程团队中验证、确认与代码审查的启示。