In this paper, we study the problem of secret language in NLP, where current language models (LMs) seem to have a hidden vocabulary that allows them to interpret absurd inputs as meaningful concepts. We investigate two research questions: ``Does the secret language phenomenon exist in different language models?'' and ``Does secret language depend on specific context?'' To answer these questions, we introduce a novel method named \textit{SecretFinding}, a gradient-based approach that can automatically discover secret languages in LMs. We conduct experiments on five representative models (Electra, ALBERT, Roberta, DistillBERT, and CLIP) finetuned on four NLP benchmarks (SST-2, MRPC, SNLI, and SQuAD) and a language-grounding benchmark (MSCOCO). Our experimental results show that even when we replace the most important words with others that are semantically dissimilar to the original words in a sentence, LMs do not consider the new sentence semantically dissimilar to the original, as the output does not change with a high probability. This phenomenon holds true across the five models and five tasks and gives a positive answer to the first research question. As for the second research question, we find that the secret language discovered by \textit{SecretFinding} is quite general and could even be transferred to other models in the black-box settings, such as GPT-3 and ChatGPT. Finally, we discuss the causes of secret language, how to eliminate it, the potential connection to memorization, and ethical implications. Examples of secret language found by SecretFinding are available on https://huggingface.co/spaces/anonymousauthors/ACL23_SecretLanguage.

翻译：在本文中，我们研究了自然语言处理中的秘密语言问题，即当前的语言模型（LMs）似乎拥有一种隐藏词汇，使其能够将荒谬的输入解释为有意义的概念。我们探讨了两个研究问题：“秘密语言现象是否存在于不同的语言模型中？”以及“秘密语言是否依赖于特定上下文？”为回答这些问题，我们提出了一种名为\textit{SecretFinding}的新方法，这是一种基于梯度的自动发现语言模型中秘密语言的方法。我们在五个代表性模型（Electra、ALBERT、Roberta、DistillBERT和CLIP）上进行实验，这些模型在四个NLP基准（SST-2、MRPC、SNLI和SQuAD）和一个语言-接地基准（MSCOCO）上进行了微调。我们的实验结果表明，即使我们将句子中最关键的词替换为与原词语义不相似的其他词，语言模型也不认为新句子与原句在语义上不相似，因为输出以高概率保持不变。这一现象在五个模型和五个任务中均成立，从而对第一个研究问题给出了肯定回答。对于第二个研究问题，我们发现由\textit{SecretFinding}发现的秘密语言具有相当的通用性，甚至可以迁移到其他黑盒设置下的模型，如GPT-3和ChatGPT。最后，我们讨论了秘密语言的成因、消除方法、与记忆化的潜在关联以及伦理影响。SecretFinding发现的秘密语言示例可在https://huggingface.co/spaces/anonymousauthors/ACL23_SecretLanguage 获取。