Knowledge workers frequently encounter repetitive web data entry tasks, like updating records or placing orders. Web automation increases productivity, but translating tasks to web actions accurately and extending to new specifications is challenging. Existing tools can automate tasks that perform the same logical trace of UI actions (e.g., input text in each field in order), but do not support tasks requiring different executions based on varied input conditions. We present DiLogics, a programming-by-demonstration system that utilizes NLP to assist users in creating web automation programs that handle diverse specifications. DiLogics first semantically segments input data to structured task steps. By recording user demonstrations for each step, DiLogics generalizes the web macros to novel but semantically similar task requirements. Our evaluation showed that non-experts can effectively use DiLogics to create automation programs that fulfill diverse input instructions. DiLogics provides an efficient, intuitive, and expressive method for developing web automation programs satisfying diverse specifications.

翻译：知识工作者经常面临重复性的网页数据输入任务，例如更新记录或下单。网页自动化能提升生产力，但将任务精准转化为网页操作并扩展至新的规范场景仍具挑战性。现有工具可自动化执行相同UI操作逻辑轨迹的任务（例如按顺序在每个字段中输入文本），但无法支持需依据不同输入条件执行差异化操作的任务。我们提出DiLogics——一种通过示范编程的系统，利用自然语言处理（NLP）辅助用户创建能处理多样化规范的网页自动化程序。DiLogics首先将输入数据语义分割为结构化任务步骤。通过记录用户针对每个步骤的演示操作，DiLogics可将网页宏推广至新颖但语义相似的任务需求。实验评估表明，非专家用户能有效利用DiLogics创建满足多样化输入指令的自动化程序。DiLogics为开发满足多样化规范的网页自动化程序提供了一种高效、直观且富有表达能力的方法。