Large Language Models (LLMs) have shown impressive performance in natural language tasks, but their outputs can exhibit undesirable attributes or biases. Existing methods for steering LLMs towards desired attributes often assume unbiased representations and rely solely on steering prompts. However, the representations learned from pre-training can introduce semantic biases that influence the steering process, leading to suboptimal results. We propose LLMGuardaril, a novel framework that incorporates causal analysis and adversarial learning to obtain unbiased steering representations in LLMs. LLMGuardaril systematically identifies and blocks the confounding effects of biases, enabling the extraction of unbiased steering representations. Additionally, it includes an explainable component that provides insights into the alignment between the generated output and the desired direction. Experiments demonstrate LLMGuardaril's effectiveness in steering LLMs towards desired attributes while mitigating biases. Our work contributes to the development of safe and reliable LLMs that align with desired attributes. We discuss the limitations and future research directions, highlighting the need for ongoing research to address the ethical implications of large language models.

翻译：大型语言模型（LLMs）在自然语言任务中展现出显著性能，但其输出可能呈现不良属性或偏见。现有引导LLMs朝向期望属性的方法通常假设表征无偏，并仅依赖提示引导。然而，预训练习得的表征可能引入语义偏见，影响引导过程并导致次优结果。我们提出LLMGuardaril——一种融合因果分析与对抗学习的新型框架，用于获取LLMs中的无偏引导表征。LLMGuardaril系统性地识别并阻断偏见的混淆效应，实现无偏引导表征的提取。此外，其可解释组件能洞察生成输出与期望方向之间的对齐程度。实验证明LLMGuardaril在引导LLMs朝向期望属性同时缓解偏见的有效性。我们的工作为开发与期望属性对齐的安全可靠LLMs做出贡献，并讨论局限性及未来研究方向，强调需持续研究以应对大型语言模型的伦理影响。