Language Models as a Service (LMaaS) offers convenient access for developers and researchers to perform inference using pre-trained language models. Nonetheless, the input data and the inference results containing private information are exposed as plaintext during the service call, leading to privacy issues. Recent studies have started tackling the privacy issue by transforming input data into privacy-preserving representation from the user-end with the techniques such as noise addition and content perturbation, while the exploration of inference result protection, namely decision privacy, is still a blank page. In order to maintain the black-box manner of LMaaS, conducting data privacy protection, especially for the decision, is a challenging task because the process has to be seamless to the models and accompanied by limited communication and computation overhead. We thus propose Instance-Obfuscated Inference (IOI) method, which focuses on addressing the decision privacy issue of natural language understanding tasks in their complete life-cycle. Besides, we conduct comprehensive experiments to evaluate the performance as well as the privacy-protection strength of the proposed method on various benchmarking tasks.

翻译：语言模型即服务（LMaaS）为开发者和研究人员提供了便捷的途径，使其能够使用预训练语言模型进行推理。然而，输入数据及包含隐私信息的推理结果在服务调用过程中以明文形式暴露，引发隐私问题。近期研究开始通过在用户端采用噪声添加和内容扰动等技术，将输入数据转换为隐私保护表示以解决隐私问题，但对推理结果保护（即决策隐私）的探索仍属空白。为维持LMaaS的黑箱模式，实施数据隐私保护（尤其是决策隐私）是一项具有挑战性的任务，因为该过程必须对模型无缝衔接，且仅产生有限的通信与计算开销。为此，我们提出实例混淆推理（IOI）方法，专注于解决自然语言理解任务在整个生命周期中的决策隐私问题。此外，我们在多种基准任务上开展了全面实验，以评估所提方法的性能及隐私保护强度。