Neural networks adapt very well to distributed and continuous representations, but struggle to generalize from small amounts of data. Symbolic systems commonly achieve data efficient generalization by exploiting modularity to benefit from local and discrete features of a representation. These features allow symbolic programs to be improved one module at a time and to experience combinatorial growth in the values they can successfully process. However, it is difficult to design a component that can be used to form symbolic abstractions and which is adequately overparametrized to learn arbitrary high-dimensional transformations. I present Graph-based Symbolically Synthesized Neural Networks (G-SSNNs), a class of neural modules that operate on representations modified with synthesized symbolic programs to include a fixed set of local and discrete features. I demonstrate that the choice of injected features within a G-SSNN module modulates the data efficiency and generalization of baseline neural models, creating predictable patterns of both heightened and curtailed generalization. By training G-SSNNs, we also derive information about desirable semantics of symbolic programs without manual engineering. This information is compact and amenable to abstraction, but can also be flexibly recontextualized for other high-dimensional settings. In future work, I will investigate data efficient generalization and the transferability of learned symbolic representations in more complex G-SSNN designs based on more complex classes of symbolic programs. Experimental code and data are available at https://github.com/shlomenu/symbolically_synthesized_networks .

翻译：神经网络非常适应分布式和连续表示，但在少量数据下的泛化能力较弱。符号系统通常通过利用模块化来受益于表示的局部和离散特征，从而实现数据高效的泛化。这些特征使得符号程序可以逐一模块地改进，并在其成功处理的值上经历组合增长。然而，设计一个既能用于形成符号抽象、又充分过参数化以学习任意高维变换的组件是困难的。本文提出了图基符号合成神经网络（G-SSNNs），这是一类神经模块，其操作基于经过合成符号程序修改的表示，从而加入一组固定的局部和离散特征。我证明，在G-SSNN模块内注入特征的选择会调节基线神经模型的数据效率和泛化能力，产生既可增强又可削弱的可预测泛化模式。通过训练G-SSNNs，我们还能在无需手动工程的情况下推导出符号程序所需的语义信息。这些信息紧凑且易于抽象，但也可灵活地重新语境化，以适应其他高维场景。在未来工作中，我将研究基于更复杂符号程序类别的G-SSNN设计中的数据高效泛化以及学习到的符号表示的可迁移性。实验代码和数据可在 https://github.com/shlomenu/symbolically_synthesized_networks 获取。