UI-to-code technology has streamlined the front-end development process, reducing repetitive tasks for engineers. prior research mainly use design prototypes as inputs, with the effectiveness of the generated code heavily dependent on these prototypes' quality, leading to compromised robustness. Moreover, these approaches also exhibit shortcomings in code quality, including issues such as disorganized UI structures and the inability to support responsive layouts. To address these challenges, we introduce Prototype2Code, which achieves end-to-end front-end code generation with business demands. For Prototype2Code, we incorporate design linting into the workflow, addressing the detection of fragmented elements and perceptual groups, enhancing the robustness of the generated outcomes. By optimizing the hierarchical structure and intelligently recognizing UI element types, Prototype2Code generates code that is more readable and structurally clearer. To meet responsive design requirements, Prototype2Code primarily supports flexbox layout model, ensuring code compatibility across various device sizes. To validate the efficacy, we compare Prototype2Code with the commercial code generation platform CodeFun and Screenshot-to-code based on GPT-4 with vision. Employing structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and mean squared error (MSE) for visual similarity assessment, Prototype2Code's rendered UI effects align most closely with the design prototypes, exhibiting the minimal errors. We also conduct a user study with five experienced front-end engineers, inviting them to review and revise code generated by the three methods. As a result, Prototype2Code surpasses other methods in readability, usability, and maintainability, better meeting the business needs of industrial development.
翻译:UI到代码技术简化了前端开发流程,减少了工程师的重复性劳动。以往研究主要将设计原型作为输入,生成代码的有效性严重依赖原型质量,导致鲁棒性不足。此外,这些方法在代码质量方面也存在缺陷,包括UI结构混乱、无法支持响应式布局等问题。为解决这些挑战,我们提出了Prototype2Code,实现了基于业务需求的端到端前端代码生成。在Prototype2Code中,我们将设计规范检查融入工作流,解决了碎片化元素和感知群的检测问题,增强了生成结果的鲁棒性。通过优化层级结构并智能识别UI元素类型,Prototype2Code生成的代码可读性更强、结构更清晰。为满足响应式设计需求,Prototype2Code主要支持flexbox布局模型,确保代码在不同设备尺寸下的兼容性。为验证有效性,我们将Prototype2Code与商业代码生成平台CodeFun及基于GPT-4视觉能力的Screenshot-to-code进行对比。采用结构相似性指数(SSIM)、峰值信噪比(PSNR)和均方误差(MSE)进行视觉相似度评估,Prototype2Code渲染的UI效果与设计原型最为接近,误差最小。我们还邀请了五位资深前端工程师开展用户研究,请他们审查并修改三种方法生成的代码。结果表明,Prototype2Code在可读性、可用性和可维护性方面均优于其他方法,能更好满足工业开发的业务需求。