Many computational tasks can be naturally expressed as a composition of a DNN followed by a program written in a traditional programming language or an API call to an LLM. We call such composites "neural programs" and focus on the problem of learning the DNN parameters when the training data consist of end-to-end input-output labels for the composite. When the program is written in a differentiable logic programming language, techniques from neurosymbolic learning are applicable, but in general, the learning for neural programs requires estimating the gradients of black-box components. We present an algorithm for learning neural programs, called ISED, that only relies on input-output samples of black-box components. For evaluation, we introduce new benchmarks that involve calls to modern LLMs such as GPT-4 and also consider benchmarks from the neurosymbolic learning literature. Our evaluation shows that for the latter benchmarks, ISED has comparable performance to state-of-the-art neurosymbolic frameworks. For the former, we use adaptations of prior work on gradient approximations of black-box components as a baseline, and show that ISED achieves comparable accuracy but in a more data- and sample-efficient manner.

翻译：许多计算任务可以自然地表达为一个深度神经网络（DNN）与一个用传统编程语言编写的程序或一个对大语言模型（LLM）的API调用的组合。我们将此类组合称为“神经程序”，并专注于在训练数据仅包含组合端到端输入输出标签的情况下学习DNN参数的问题。当程序采用可微逻辑编程语言编写时，神经符号学习技术是适用的，但一般而言，神经程序的学习需要估计黑盒组件的梯度。我们提出了一种名为ISED的学习神经程序的算法，该算法仅依赖于黑盒组件的输入输出样本。为进行评估，我们引入了涉及调用现代LLM（如GPT-4）的新基准测试，并考虑了神经符号学习文献中的现有基准。我们的评估表明，对于后者，ISED的性能与最先进的神经符号框架相当。对于前者，我们采用先前关于黑盒组件梯度近似的工作作为基线，并证明ISED在达到相当精度的同时，具有更高的数据和样本效率。