We discuss a federated learned compression problem, where the goal is to learn a compressor from real-world data which is scattered across clients and may be statistically heterogeneous, yet share a common underlying representation. We propose a distributed source model that encompasses both characteristics, and naturally suggests a compressor architecture that uses analysis and synthesis transforms shared by clients. Inspired by personalized federated learning methods, we employ an entropy model that is personalized to each client. This allows for a global latent space to be learned across clients, and personalized entropy models that adapt to the clients' latent distributions. We show empirically that this strategy outperforms solely local methods, which indicates that learned compression also benefits from a shared global representation in statistically heterogeneous federated settings.

翻译：我们讨论了一个联邦学习压缩问题，其目标是从散布在客户端且可能具有统计异质性但共享共同底层表示的真实世界数据中学习压缩器。我们提出了一种涵盖这两个特性的分布式信源模型，该模型自然提示了一种使用客户端共享的分析与合成变换的压缩器架构。受个性化联邦学习方法的启发，我们采用了一种针对每个客户端进行个性化的熵模型。这使得可以在客户端之间学习全局潜在空间，并采用适应客户端潜在分布的个性化熵模型。我们的实验表明，该策略优于仅限本地的方法，这表明在统计异质性的联邦设置中，学习压缩也能从共享的全局表示中获益。