Neural networks adapt very well to distributed and continuous representations, but struggle to learn and generalize from small amounts of data. Symbolic systems commonly achieve data efficient generalization by exploiting modularity to benefit from local and discrete features of a representation. These features allow symbolic programs to be improved one module at a time and to experience combinatorial growth in the values they can successfully process. However, it is difficult to design components that can be used to form symbolic abstractions and which are highly-overparametrized like neural networks, as the adjustment of parameters makes the semantics of modules unstable. I present Graph-based Symbolically Synthesized Neural Networks (G-SSNNs), a form of neural network whose topology and parameters are informed by the output of a symbolic program. I demonstrate that by developing symbolic abstractions at a population level, and applying gradient-based optimization to such neural models at an individual level, I can elicit reliable patterns of improved generalization with small quantities of data known to contain local and discrete features. The paradigm embodied by G-SSNNs offers a route towards the communal development of compact and composable abstractions which can be flexibly repurposed for a variety of tasks and high-dimensional media. In future work, I hope to pursue these benefits by exploring more ambitious G-SSNN designs based on more complex classes of symbolic programs. The code and data associated with the reported results are publicly available at https://github.com/shlomenu/symbolically_synthesized_networks .

翻译：神经网络非常适应分布式和连续表示，但在少量数据的学习和泛化方面存在困难。符号系统通常通过利用模块化来从局部和离散特征中受益，从而实现数据高效的泛化。这些特征使得符号程序能够逐个模块地改进，并在可成功处理的值上实现组合增长。然而，设计可用于形成符号抽象且像神经网络一样高度过参数化的组件很困难，因为参数的调整使得模块的语义不稳定。本文提出了基于图结构的符号合成神经网络（G-SSNNs），这是一种拓扑结构和参数受符号程序输出影响的神经网络形式。通过开发种群级别的符号抽象，并在个体级别对这些神经模型应用基于梯度的优化，我们能够在已知包含局部和离散特征的小量数据中，引出更可靠的泛化改进模式。G-SSNNs所体现的范式为社区开发紧凑且可组合的抽象提供了一条途径，这些抽象可灵活地重新用于多种任务和高维媒体。在未来的工作中，我计划通过探索基于更复杂符号程序类的G-SSNN设计来追求这些优势。相关结果所涉及的代码和数据已在https://github.com/shlomenu/symbolically_synthesized_networks 公开提供。