The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics.
翻译:随着大语言模型(LLMs)的兴起,通过Vercel V0等工具实现前端界面创建已变得更为便捷,但设计质量(如可访问性、可用性)问题也随之凸显。现有解决方案常因其关注点局限、泛化能力不足或数据依赖性等缺陷,难以有效应对这些复杂挑战。此外,当前尚无研究系统评估LLM生成的UI设计质量。本研究提出DesignRepair——一种新颖的双流设计准则感知系统,能够从代码层面与渲染页面层面对UI设计质量问题进行检测与修复。我们采用成熟且广泛使用的Material Design作为指导该过程的知识库。具体而言,我们首先构建了综合性知识库,将Google Material Design准则编码为底层组件知识库与高层系统设计知识库。随后,DesignRepair利用LLM提取关键组件,并采用Playwright工具进行精确的页面分析,将分析结果与既有知识库进行对齐。最后,我们通过检索增强生成技术整合GPT-4等先进LLM,采用分治策略对前端代码进行整体优化与修复。大量实验验证了本方法的有效性与实用性,结果表明其在设计准则遵循度、可访问性及用户体验指标方面均实现显著提升。