Present-day graphical user interfaces (GUIs) exhibit diverse arrangements of text, graphics, and interactive elements such as buttons and menus, but representations of GUIs have not kept up. They do not encapsulate both semantic and visuo-spatial relationships among elements. To seize machine learning's potential for GUIs more efficiently, Graph4GUI exploits graph neural networks to capture individual elements' properties and their semantic-visuo-spatial constraints in a layout. The learned representation demonstrated its effectiveness in multiple tasks, especially generating designs in a challenging GUI autocompletion task, which involved predicting the positions of remaining unplaced elements in a partially completed GUI. The new model's suggestions showed alignment and visual appeal superior to the baseline method and received higher subjective ratings for preference. Furthermore, we demonstrate the practical benefits and efficiency advantages designers perceive when utilizing our model as an autocompletion plug-in.

翻译：当前的图形用户界面（GUI）展示了文本、图形以及按钮、菜单等交互元素的多样化排列，但GUI的表示方法未能跟上发展，未能同时封装元素间的语义关系和视觉-空间关系。为更有效地发挥机器学习在GUI领域的潜力，Graph4GUI利用图神经网络捕捉布局中单个元素的属性及其语义-视觉-空间约束。习得的表示在多项任务中证明了其有效性，尤其在具有挑战性的GUI自动补全任务中——该任务涉及预测部分完成GUI中剩余未放置元素的位置。新模型的建议在排列对齐和视觉吸引力上均优于基线方法，并在主观偏好评分中获得更高评价。此外，我们展示了设计师将其作为自动补全插件使用时感知到的实际效能优势与效率提升。