This paper pursues the insight that language models naturally enable an intelligent variation operator similar in spirit to evolutionary crossover. In particular, language models of sufficient scale demonstrate in-context learning, i.e. they can learn from associations between a small number of input patterns to generate outputs incorporating such associations (also called few-shot prompting). This ability can be leveraged to form a simple but powerful variation operator, i.e. to prompt a language model with a few text-based genotypes (such as code, plain-text sentences, or equations), and to parse its corresponding output as those genotypes' offspring. The promise of such language model crossover (which is simple to implement and can leverage many different open-source language models) is that it enables a simple mechanism to evolve semantically-rich text representations (with few domain-specific tweaks), and naturally benefits from current progress in language models. Experiments in this paper highlight the versatility of language-model crossover, through evolving binary bit-strings, sentences, equations, text-to-image prompts, and Python code. The conclusion is that language model crossover is a promising method for evolving genomes representable as text.

翻译：本文探讨了一种洞见：语言模型能够自然地实现一种智能变异算子，其精神类似于进化计算中的交叉操作。具体而言，规模足够的语言模型展现出上下文学习能力，即它们可以从少量输入模式之间的关联中学习，并生成融合此类关联的输出（也称为少样本提示）。这种能力可用于构建一个简单而强大的变异算子——即用少数基于文本的基因型（如代码、纯文本句子或方程）对语言模型进行提示，并将其对应输出解析为这些基因型的后代。语言模型交叉（易于实现，并可利用多种开源语言模型）的潜力在于：它为进化语义丰富的文本表示提供了一种简单机制（仅需少量领域特定调整），并能自然受益于语言模型的当前进展。本文通过进化二进制比特串、句子、方程、文本到图像提示及Python代码等实验，凸显了语言模型交叉的通用性。结论表明，语言模型交叉是一种有望进化可表示为文本的基因组的方法。