User feedback is crucial for the evolution of mobile apps. However, research suggests that users tend to submit uninformative, vague, or destructive feedback. Unlike recent AI4SE approaches that focus on generating code and other development artifacts, our work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app. We propose LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives, from which the user can easily choose their preferred option. To evaluate LikeThis!, we first conducted a model benchmarking study based on a public dataset of carefully critiqued UI designs. The results show that GPT-Image-1 significantly outperformed three other state-of-the-art image generation models in improving the designs to address UI issues while keeping the fidelity and without introducing new issues. An intermediate step in LikeThis! is to generate a solution specification before sketching the design as a key to achieving effective improvement. Second, we conducted a user study with 10 production apps, where 15 users used LikeThis! to submit their feedback on encountered issues. Later, the developers of the apps assessed the understandability and actionability of the feedback with and without generated improvements. The results show that our approach helps generate better feedback from both user and developer perspectives, paving the way for AI-assisted user-developer collaboration.
翻译:用户反馈对于移动应用的演进至关重要。然而,研究表明用户倾向于提交信息量不足、模糊或破坏性的反馈。与近期专注于生成代码及其他开发产物的AI4SE方法不同,我们的工作旨在赋能用户提交更优质、更具建设性的界面反馈,并提供具体的应用改进建议。我们提出了LikeThis!,一种基于生成式人工智能的方法,该方法接收用户评论及对应截图,即时生成多种改进方案,用户可从中轻松选择其偏好选项。为评估LikeThis!,我们首先基于经严格评审的界面设计公共数据集进行了模型基准测试。结果表明,在改进设计以解决界面问题、保持保真度且不引入新问题方面,GPT-Image-1显著优于其他三种前沿图像生成模型。LikeThis!的关键中间步骤是在绘制设计草图前生成解决方案规范,这是实现有效改进的核心。其次,我们针对10款生产级应用开展了用户研究,15名用户使用LikeThis!就遭遇的问题提交反馈。随后,应用开发者从可理解性与可操作性两个维度,评估了附带生成改进方案与未附带改进方案的反馈质量。结果表明,我们的方法从用户和开发者双重视角均有助于生成更优质的反馈,为人工智能辅助的用户-开发者协作开辟了新路径。