The generalization error of a classifier is related to the complexity of the set of functions among which the classifier is chosen. We study a family of low-complexity classifiers consisting of thresholding a random one-dimensional feature. The feature is obtained by projecting the data on a random line after embedding it into a higher-dimensional space parametrized by monomials of order up to k. More specifically, the extended data is projected n-times and the best classifier among those n, based on its performance on training data, is chosen. We show that this type of classifier is extremely flexible, as it is likely to approximate, to an arbitrary precision, any continuous function on a compact set as well as any boolean function on a compact set that splits the support into measurable subsets. In particular, given full knowledge of the class conditional densities, the error of these low-complexity classifiers would converge to the optimal (Bayes) error as k and n go to infinity. On the other hand, if only a training dataset is given, we show that the classifiers will perfectly classify all the training points as k and n go to infinity. We also bound the generalization error of our random classifiers. In general, our bounds are better than those for any classifier with VC dimension greater than O (ln n) . In particular, our bounds imply that, unless the number of projections n is extremely large, there is a significant advantageous gap between the generalization error of the random projection approach and that of a linear classifier in the extended space. Asymptotically, as the number of samples approaches infinity, the gap persists for any such n. Thus, there is a potentially large gain in generalization properties by selecting parameters at random, rather than optimization.

翻译：分类器的泛化误差与函数集的复杂度相关，该分类器即从中选取。本文研究一类低复杂度分类器，其基于随机一维特征的阈值化。该特征通过将数据投影到随机直线上获得，投影前需将数据嵌入由阶数不超过k的单项式参数化的高维空间。具体而言，我们将扩展数据投影n次，并基于训练集上的表现从中选取最优分类器。研究表明，这类分类器具有极强的灵活性：对于紧集上的任意连续函数以及将支撑集划分为可测子集的布尔函数，其均能以任意精度逼近。特别地，若完全掌握类条件密度，则当k和n趋于无穷时，这些低复杂度分类器的误差将收敛至最优（贝叶斯）误差。另一方面，若仅给定训练数据集，我们证明当k和n趋于无穷时，分类器能完美分类所有训练点。同时，我们给出了随机分类器的泛化误差界。总体而言，我们的误差界优于任何VC维大于O(ln n)的分类器。特别地，该误差界表明：除非投影数n极大，否则随机投影方法在泛化误差上显著优于扩展空间中的线性分类器。当样本量趋于无穷时，该优势对于任意有限n持续存在。因此，参数随机选取相较于参数优化在泛化性能方面可能带来显著增益。