In this study, we tackle a growing concern around the safety and ethical use of large language models (LLMs). Despite their potential, these models can be tricked into producing harmful or unethical content through various sophisticated methods, including 'jailbreaking' techniques and targeted manipulation. Our work zeroes in on a specific issue: to what extent LLMs can be led astray by asking them to generate responses that are instruction-centric such as a pseudocode, a program or a software snippet as opposed to vanilla text. To investigate this question, we introduce TechHazardQA, a dataset containing complex queries which should be answered in both text and instruction-centric formats (e.g., pseudocodes), aimed at identifying triggers for unethical responses. We query a series of LLMs -- Llama-2-13b, Llama-2-7b, Mistral-V2 and Mistral 8X7B -- and ask them to generate both text and instruction-centric responses. For evaluation we report the harmfulness score metric as well as judgements from GPT-4 and humans. Overall, we observe that asking LLMs to produce instruction-centric responses enhances the unethical response generation by ~2-38% across the models. As an additional objective, we investigate the impact of model editing using the ROME technique, which further increases the propensity for generating undesirable content. In particular, asking edited LLMs to generate instruction-centric responses further increases the unethical response generation by ~3-16% across the different models.

翻译：本研究聚焦于大型语言模型（LLMs）的安全与伦理使用这一日益严峻的议题。尽管LLMs潜力巨大，但通过各类复杂手段（包括“越狱”技术与定向操控），仍可诱导其生成有害或不道德内容。我们的工作针对一个特定问题展开：当要求LLMs生成指令中心型回答（如伪代码、程序或代码片段）而非纯文本时，模型在多大程度上可能被误导。为探究此问题，我们构建了TechHazardQA数据集，其中包含需以文本与指令中心格式（如伪代码）同时作答的复杂查询，旨在识别诱发不道德响应的触发条件。我们测试了Llama-2-13b、Llama-2-7b、Mistral-V2及Mistral 8X7B等系列LLMs，要求其分别生成文本与指令中心型回答。评估方面，我们采用了有害性评分指标，并引入GPT-4与人工评判。总体而言，我们发现要求LLMs生成指令中心型回答会使其不道德响应生成能力提升约2-38%。作为额外目标，我们还研究了基于ROME技术的模型编辑所带来的影响——该操作进一步增加了有害内容的生成倾向。具体而言，经编辑的LLMs在被要求生成指令中心型回答时，不道德响应生成率在不同模型间进一步提升了约3-16%。