The rise of Large Language Models (LLMs) has streamlined frontend interface creation through tools like Vercel's V0, yet surfaced challenges in design quality (e.g., accessibility, and usability). Current solutions, often limited by their focus, generalisability, or data dependency, fall short in addressing these complexities. Moreover, none of them examine the quality of LLM-generated UI design. In this work, we introduce DesignRepair, a novel dual-stream design guideline-aware system to examine and repair the UI design quality issues from both code aspect and rendered page aspect. We utilised the mature and popular Material Design as our knowledge base to guide this process. Specifically, we first constructed a comprehensive knowledge base encoding Google's Material Design principles into low-level component knowledge base and high-level system design knowledge base. After that, DesignRepair employs a LLM for the extraction of key components and utilizes the Playwright tool for precise page analysis, aligning these with the established knowledge bases. Finally, we integrate Retrieval-Augmented Generation with state-of-the-art LLMs like GPT-4 to holistically refine and repair frontend code through a strategic divide and conquer approach. Our extensive evaluations validated the efficacy and utility of our approach, demonstrating significant enhancements in adherence to design guidelines, accessibility, and user experience metrics.
翻译:随着大语言模型(LLMs)的兴起,通过Vercel V0等工具实现前端界面创建已变得更为便捷,但也暴露出设计质量(如可访问性、可用性)方面的挑战。现有解决方案往往受限于其关注点、泛化能力或数据依赖性,难以应对这些复杂问题。此外,尚无研究系统评估LLM生成的UI设计质量。本文提出DesignRepair,一种新颖的双流设计准则感知系统,能够从代码层面和渲染页面层面对UI设计质量问题进行检测与修复。我们采用成熟且广泛应用的Material Design作为指导该过程的知识库。具体而言,我们首先构建了综合性知识库,将Google的Material Design原则编码为底层组件知识库和高级系统设计知识库。随后,DesignRepair利用LLM提取关键组件,并采用Playwright工具进行精确的页面分析,将分析结果与已建立的知识库进行对齐。最后,我们通过检索增强生成技术结合GPT-4等先进LLM,采用分治策略对前端代码进行整体优化与修复。大量实验验证了本方法的有效性与实用性,结果表明其在设计准则遵循度、可访问性及用户体验指标方面均实现显著提升。