We present a simple proof for the benefit of depth in multi-layer feedforward network with rectified activation ("depth separation"). Specifically we present a sequence of classification problems indexed by $m$ such that (a) for any fixed depth rectified network there exist an $m$ above which classifying problem $m$ correctly requires exponential number of parameters (in $m$); and (b) for any problem in the sequence, we present a concrete neural network with linear depth (in $m$) and small constant width ($\leq 4$) that classifies the problem with zero error. The constructive proof is based on geometric arguments and a space folding construction. While stronger bounds and results exist, our proof uses substantially simpler tools and techniques, and should be accessible to undergraduate students in computer science and people with similar backgrounds.

翻译：我们提出一个简单的证据来证明多层向外输送网络的深度,并纠正激活(“深度分离 ” ) 。 具体而言,我们提出了一系列分类问题,其指数为百万美元,这样:(a) 对于任何固定深度纠正网络,上面有1百万美元,对问题进行正确分类需要指数数参数(单位:百万美元);(b) 对于任何序列中的任何问题,我们提出一个具体神经网络,其线性深度(单位:百万美元)和小的恒定宽度(单位:4美元),将问题分为零错误。建设性证据以几何参数和空间折叠结构为基础。尽管存在更强的界限和结果,但我们的证据使用简单得多的工具和技术,并且应该让计算机科学的本科生和背景相似的人使用。