Knowledge workers frequently encounter repetitive web data entry tasks, like updating records or placing orders. Web automation increases productivity, but translating tasks to web actions accurately and extending to new specifications is challenging. Existing tools can automate tasks that perform the same logical trace of UI actions (e.g., input text in each field in order), but do not support tasks requiring different executions based on varied input conditions. We present DiLogics, a programming-by-demonstration system that utilizes NLP to assist users in creating web automation programs that handle diverse specifications. DiLogics first semantically segments input data to structured task steps. By recording user demonstrations for each step, DiLogics generalizes the web macros to novel but semantically similar task requirements. Our evaluation showed that non-experts can effectively use DiLogics to create automation programs that fulfill diverse input instructions. DiLogics provides an efficient, intuitive, and expressive method for developing web automation programs satisfying diverse specifications.

翻译：知识工作者经常遇到重复性的网页数据输入任务，如更新记录或下订单。网页自动化提高了生产力，但将任务精确转化为网页操作并扩展到新规范具有挑战性。现有工具能够自动化执行相同UI操作逻辑轨迹的任务（例如，按顺序在每个字段中输入文本），但不支持基于不同输入条件需要不同执行流程的任务。我们提出了DiLogics，一个通过演示编程的系统，利用自然语言处理帮助用户创建能处理多样化规范的网页自动化程序。DiLogics首先将输入数据语义分割为结构化任务步骤。通过记录每个步骤的用户演示，DiLogics将网页宏推广到新颖但语义相似的任务需求。我们的评估表明，非专家用户能够有效使用DiLogics创建满足多样化输入指令的自动化程序。DiLogics为开发满足多样化规范的网页自动化程序提供了一种高效、直观且富有表现力的方法。