Dropout is a widely used regularization trick to resolve the overfitting issue in large feedforward neural networks trained on a small dataset, which performs poorly on the held-out test subset. Although the effectiveness of this regularization trick has been extensively studied for convolutional neural networks, there is a lack of analysis of it for unsupervised models and in particular, VAE-based neural topic models. In this paper, we have analyzed the consequences of dropout in the encoder as well as in the decoder of the VAE architecture in three widely used neural topic models, namely, contextualized topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly available datasets. We characterize the dropout effect on these models in terms of the quality and predictive performance of the generated topics.

翻译：Dropout是一种广泛使用的正则化技巧，用于解决在小型数据集上训练的大型前馈神经网络的过拟合问题，而这类网络在保留测试子集上表现不佳。尽管Dropout在卷积神经网络中的有效性已得到广泛研究，但缺乏对它在无监督模型，特别是基于VAE的神经主题模型中的分析。本文针对三种广泛使用的神经主题模型——即上下文主题模型（CTM）、ProdLDA和嵌入式主题模型（ETM），在四个公开可用数据集上分析了Dropout在VAE架构编码器和解码器中的影响。我们从生成主题的质量和预测性能两方面刻画了Dropout对这些模型的影响。