Injecting structure into neural networks enables learning functions that satisfy invariances with respect to subsets of inputs. For instance, when learning generative models using neural networks, it is advantageous to encode the conditional independence structure of observed variables, often in the form of Bayesian networks. We propose the Structured Neural Network (StrNN), which injects structure through masking pathways in a neural network. The masks are designed via a novel relationship we explore between neural network architectures and binary matrix factorization, to ensure that the desired independencies are respected. We devise and study practical algorithms for this otherwise NP-hard design problem based on novel objectives that control the model architecture. We demonstrate the utility of StrNN in three applications: (1) binary and Gaussian density estimation with StrNN, (2) real-valued density estimation with Structured Autoregressive Flows (StrAFs) and Structured Continuous Normalizing Flows (StrCNF), and (3) interventional and counterfactual analysis with StrAFs for causal inference. Our work opens up new avenues for learning neural networks that enable data-efficient generative modeling and the use of normalizing flows for causal effect estimation.

翻译：将结构注入神经网络可使其学习满足输入子集不变性的函数。例如，在使用神经网络学习生成模型时，编码观测变量的条件独立结构（通常以贝叶斯网络形式呈现）具有显著优势。本文提出结构化神经网络（StrNN），通过掩码路径将结构注入神经网络。我们利用神经网络架构与二元矩阵分解之间新发现的关联性设计掩码，确保所期望的独立性得以保持。针对这一原本为NP难问题的网络架构设计，我们基于控制模型架构的新颖目标函数提出了实用算法并进行深入研究。通过三个应用场景展示StrNN的效用：（1）基于StrNN的二元及高斯密度估计；（2）基于结构化自回归流（StrAFs）与结构化连续正则化流（StrCNF）的实值密度估计；（3）基于StrAFs的干预与反事实分析在因果推断中的应用。本研究为学习支持数据高效生成建模的神经网络，以及利用正则化流进行因果效应估计开辟了新途径。