To design with AI models, user experience (UX) designers must assess the fit between the model and user needs. Based on user research, they need to contextualize the model's behavior and potential failures within their product-specific data instances and user scenarios. However, our formative interviews with ten UX professionals revealed that such a proactive discovery of model limitations is challenging and time-intensive. Furthermore, designers often lack technical knowledge of AI and accessible exploration tools, which challenges their understanding of model capabilities and limitations. In this work, we introduced a failure-driven design approach to AI, a workflow that encourages designers to explore model behavior and failure patterns early in the design process. The implementation of fAIlureNotes, a designer-centered failure exploration and analysis tool, supports designers in evaluating models and identifying failures across diverse user groups and scenarios. Our evaluation with UX practitioners shows that fAIlureNotes outperforms today's interactive model cards in assessing context-specific model performance.

翻译：为与AI模型协同设计，用户体验设计师必须评估模型与用户需求之间的匹配度。基于用户研究，他们需要将模型行为及潜在失败模式置于具体产品数据实例和用户场景中进行情境化分析。然而，我们对十位用户体验专业人员的形成性访谈揭示，这种主动发现模型局限性的过程既困难又耗时。此外，设计师常缺乏AI技术知识与可用的探索工具，这阻碍了他们理解模型能力与局限性。在本研究中，我们提出了一种面向失败的AI设计方法，该工作流鼓励设计师在设计早期探索模型行为与失败模式。fAIlureNotes作为以设计师为中心的失败探索与分析工具的实现，支持设计师评估模型并识别不同用户群体与场景中的失败模式。与用户体验从业者的评估表明，fAIlureNotes在评估情境化模型性能方面优于现今的交互式模型卡片。