The integration of Large Language Models (LLMs) into various software applications raises concerns about their potential biases. Typically, those models are trained on a vast amount of data scrapped from forums, websites, social media and other internet sources, which may instill harmful and discriminating behavior into the model. To address this issue, we present LangBiTe, a testing platform to systematically assess the presence of biases within an LLM. LangBiTe enables development teams to tailor their test scenarios, and automatically generate and execute the test cases according to a set of user-defined ethical requirements. Each test consists of a prompt fed into the LLM and a corresponding test oracle that scrutinizes the LLM's response for the identification of biases. LangBite provides users with the bias evaluation of LLMs, and end-to-end traceability between the initial ethical requirements and the insights obtained.

翻译：大型语言模型（LLM）在各类软件应用中的集成引发了对其潜在偏见的担忧。这类模型通常基于从论坛、网站、社交媒体及其他网络来源抓取的海量数据进行训练，可能会向模型中植入有害且具有歧视性的行为。为解决此问题，我们提出了LangBiTe——一个系统性评估LLM中是否存在偏见的测试平台。LangBiTe支持开发团队定制测试场景，并根据用户定义的伦理需求自动生成并执行测试用例。每个测试由输入至LLM的提示词与相应的测试预言组成，该预言用于审查LLM的响应以识别偏见。LangBiTe为用户提供LLM的偏见评估结果，并实现从初始伦理需求到所得洞察的端到端可追溯性。