We describe a simple and effective method (Spectral Attribute removaL; SAL) to remove private or guarded information from neural representations. Our method uses matrix decomposition to project the input representations into directions with reduced covariance with the guarded information rather than maximal covariance as factorization methods normally use. We begin with linear information removal and proceed to generalize our algorithm to the case of nonlinear information removal using kernels. Our experiments demonstrate that our algorithm retains better main task performance after removing the guarded information compared to previous work. In addition, our experiments demonstrate that we need a relatively small amount of guarded attribute data to remove information about these attributes, which lowers the exposure to sensitive data and is more suitable for low-resource scenarios. Code is available at https://github.com/jasonshaoshun/SAL.

翻译：我们提出了一种简单且有效的方法（谱属性移除，简称SAL），用于从神经表征中移除隐私或受保护信息。该方法利用矩阵分解，将输入表征投影到与受保护信息协方差减小的方向上，而非如传统分解方法那样最大化协方差。我们从线性信息移除入手，进而推广至基于核的非线性信息移除算法。实验表明，与先前工作相比，我们的算法在移除受保护信息后能保持更好的主任务性能。此外，实验证明，我们仅需相对少量的受保护属性数据即可移除这些属性信息，从而降低对敏感数据的暴露，更适用于低资源场景。代码已开源至 https://github.com/jasonshaoshun/SAL。