Disentangling sentence representations over continuous spaces can be a critical process in improving interpretability and semantic control by localising explicit generative factors. Such process confers to neural-based language models some of the advantages that are characteristic of symbolic models, while keeping their flexibility. This work presents a methodology for disentangling the hidden space of a BERT-GPT2 autoencoder by transforming it into a more separable semantic space with the support of a flow-based invertible neural network (INN). Experimental results indicate that the INN can transform the distributed hidden space into a better semantically disentangled latent space, resulting in better interpretability and controllability, when compared to recent state-of-the-art models.

翻译：将连续空间中的句子表示解耦，能够通过定位显式生成因子来提升可解释性和语义控制能力。这一过程赋予基于神经的语言模型符号模型的部分优势，同时保持其灵活性。本文提出一种方法，通过基于流的可逆神经网络（INN）将BERT-GPT2自编码器的隐藏空间转化为更可分离的语义空间，从而实现解耦。实验结果表明，与近期最优模型相比，INN能够将分布式隐藏空间转化为语义上更优的解耦潜在空间，从而获得更好的可解释性和可控性。