In this study, we tackle a growing concern around the safety and ethical use of large language models (LLMs). Despite their potential, these models can be tricked into producing harmful or unethical content through various sophisticated methods, including 'jailbreaking' techniques and targeted manipulation. Our work zeroes in on a specific issue: to what extent LLMs can be led astray by asking them to generate responses that are instruction-centric such as a pseudocode, a program or a software snippet as opposed to vanilla text. To investigate this question, we introduce TechHazardQA, a dataset containing complex queries which should be answered in both text and instruction-centric formats (e.g., pseudocodes), aimed at identifying triggers for unethical responses. We query a series of LLMs -- Llama-2-13b, Llama-2-7b, Mistral-V2 and Mistral 8X7B -- and ask them to generate both text and instruction-centric responses. For evaluation we report the harmfulness score metric as well as judgements from GPT-4 and humans. Overall, we observe that asking LLMs to produce instruction-centric responses enhances the unethical response generation by ~2-38% across the models. As an additional objective, we investigate the impact of model editing using the ROME technique, which further increases the propensity for generating undesirable content. In particular, asking edited LLMs to generate instruction-centric responses further increases the unethical response generation by ~3-16% across the different models.

翻译：本研究针对大型语言模型（LLMs）的安全与伦理使用这一日益增长的关切展开探讨。尽管这些模型潜力巨大，但它们可能通过各类复杂手段（包括"越狱"技术与针对性操纵）被诱导生成有害或不道德的内容。我们聚焦于一个特定问题：当要求LLMs生成指令导向的回应（如伪代码、程序或软件片段）而非普通文本时，模型在实际运行中会被误导到何种程度？为探究此问题，我们提出了TechHazardQA数据集，其中包含需要以文本和指令导向格式（如伪代码）回答的复杂查询，旨在识别引发不道德回应的触发因素。我们选取了Llama-2-13b、Llama-2-7b、Mistral-V2及Mistral 8X7B系列LLMs进行查询，要求其同时生成文本回应与指令导向回应。在评估中，我们报告了危害性评分指标，并参考了GPT-4及人工判断结果。总体而言，我们观察到要求LLMs生成指令导向回应会使不道德回应生成率提升约2-38%。此外，我们还探讨了采用ROME技术进行模型编辑的影响——该操作进一步提高了生成不良内容的倾向性。具体而言，要求经编辑后的LLMs生成指令导向回应，会使不同模型的不道德回应生成率额外增加约3-16%。