Compiler correctness is a cornerstone of reliable software development. However, systematic testing of compilers is infeasible, given the vast space of possible programs and the complexity of modern programming languages. In this context, differential testing offers a practical methodology as it addresses the oracle problem by comparing the output of alternative compilers given the same set of programs as input. In this paper, we investigate the effectiveness of differential testing in finding bugs within the Kotlin compilers developed at JetBrains. We propose a black-box generative approach that creates input programs for the K1 and K2 compilers. First, we build workable models of Kotlin semantic (semantic interface) and syntactic (enriched context-free grammar) language features, which are subsequently exploited to generate random code snippets. Second, we extend random sampling by introducing two genetic algorithms (GAs) that aim to generate more diverse input programs. Our case study shows that the proposed approach effectively detects bugs in K1 and K2; these bugs have been confirmed and (some) fixed by JetBrains developers. While we do not observe a significant difference w.r.t. the number of defects uncovered by the different search algorithms, random search and GAs are complementary as they find different categories of bugs. Finally, we provide insights into the relationships between the size, complexity, and fault detection capability of the generated input programs.

翻译：编译器正确性是可靠软件开发的基石。然而，由于可能程序的空间极其庞大且现代编程语言复杂，对编译器进行系统性测试并不可行。在此背景下，差异测试提供了一种实用方法，它通过比较同一组程序作为输入时不同编译器的输出来解决预言机问题。本文研究了差异测试在发现JetBrains开发的Kotlin编译器中的缺陷的有效性。我们提出了一种黑盒生成方法，为K1和K2编译器创建输入程序。首先，我们构建了Kotlin语义（语义接口）和句法（增强的上下文无关文法）语言特征的可工作模型，随后利用这些模型生成随机代码片段。其次，我们引入两种遗传算法（GA）扩展随机采样，旨在生成更多样化的输入程序。我们的案例研究表明，所提出的方法能有效检测K1和K2中的缺陷；这些缺陷已得到JetBrains开发人员的确认，并且（部分）已被修复。虽然我们未观察到不同搜索算法在发现的缺陷数量上存在显著差异，但随机搜索和遗传算法具有互补性，因为它们能发现不同类型的缺陷。最后，我们深入分析了生成的输入程序的大小、复杂度与缺陷检测能力之间的关系。