Recent literature on unsupervised learning focused on designing structural priors with the aim of learning meaningful features, but without considering the description length of the representations. In this thesis, first we introduce the $\varphi$ metric that evaluates unsupervised models based on their reconstruction accuracy and the degree of compression of their internal representations. We then present and define two activation functions (Identity, ReLU) as base of reference and three sparse activation functions (top-k absolutes, Extrema-Pool indices, Extrema) as candidate structures that minimize the previously defined metric $\varphi$. We lastly present Sparsely Activated Networks (SANs) that consist of kernels with shared weights that, during encoding, are convolved with the input and then passed through a sparse activation function. During decoding, the same weights are convolved with the sparse activation map and subsequently the partial reconstructions from each weight are summed to reconstruct the input. We compare SANs using the five previously defined activation functions on a variety of datasets (Physionet, UCI-epilepsy, MNIST, FMNIST) and show that models that are selected using $\varphi$ have small description representation length and consist of interpretable kernels.

翻译：有关未经监督的近期学习文献侧重于设计结构前程,目的是学习有意义的特征,但是没有考虑演示的描述长度。在此论文中,我们首先引入了美元标准,根据重建准确度和内部陈述压缩程度对未经监督的模型进行评估。然后我们提出并定义了两个启动功能(身份、RELU)作为参考基础,以及三个稀疏的启动功能(顶-k绝对值、Extrema-pool指数、Extrema)作为候选结构,以尽量减少先前定义的美元。我们最后展示了SANs,这些网络由共享权重的内核组成,在编码过程中,与输入混杂在一起,然后通过稀疏的激活功能传递。在解码过程中,同样的权重与稀疏的激活地图混杂在一起,然后对每个重量的部分重建进行总结,以重建投入。我们用先前定义的五个激活功能对SANs进行了比较,在各种数据集(Physituet、Sparly 激活网络)上,由共享权重的内核内层网络组成,在编码过程中,与输入FMIIST(MIS) 和经选的缩缩的模型。