Due to the development of pre-trained language models, automated code generation techniques have shown great promise in recent years. However, the generated code is difficult to meet the syntactic constraints of the target language, especially in the case of Turducken-style code, where declarative code snippets are embedded within imperative programs. In this study, we summarize the lack of syntactic constraints into three significant challenges: (1) the efficient representation of syntactic constraints, (2) the effective integration of syntactic information, and (3) the scalable syntax-first decoding algorithm. To address these challenges, we propose a syntax-guided multi-task learning approach TurduckenGen. Specifically, we first explicitly append the type information to the code tokens to capture the representation of syntactic constraints. Then we formalize code generation with syntactic constraint representation as an auxiliary task to enable the model to learn the syntactic constraints of the code. Finally, the syntactically correct code is selected accurately from the multiple candidates with the help of the compiler feedback. Extensive experiments and comprehensive analysis demonstrate the effectiveness and general applicability of our approach after being compared with six state-of-the-art baselines on two Turducken-style code datasets. Finally, we conducted a human study and found the code quality generated by our approach is better than baselines in terms of code readability and semantic similarity.

翻译：得益于预训练语言模型的发展，自动化代码生成技术近年来展现出巨大潜力。然而，生成的代码难以满足目标语言的语法约束，尤其是在Turducken风格代码（即声明式代码片段嵌入命令式程序中的情况）中。本研究将语法约束缺失归纳为三个关键挑战：(1) 语法约束的高效表示，(2) 语法信息的有效融合，以及(3) 可扩展的语法优先解码算法。为应对这些挑战，我们提出了一种语法引导的多任务学习方法TurduckenGen。具体而言，我们首先将类型信息显式附加至代码词元以捕获语法约束的表示；随后将带语法约束表示的代码生成形式化为辅助任务，使模型习得代码的语法约束；最后借助编译器反馈从多个候选代码中精准选择语法正确的代码。在两类Turducken风格代码数据集上与六种最先进基线方法的对比实验及全面分析表明，本方法具有有效性与通用适用性。此外，通过人工评估发现，本方法生成的代码在可读性与语义相似度方面均优于基线方法。