Now that AI-driven moderation has become pervasive in everyday life, we often hear claims that "the AI is biased". While this is often said jokingly, the light-hearted remark reflects a deeper concern. How can we be certain that an online post flagged as "inappropriate" was not simply the victim of a biased algorithm? This paper investigates this problem using a dual approach. First, I conduct a quantitative benchmark of a widely used toxicity model (unitary/toxic-bert) to measure performance disparity between text in African-American English (AAE) and Standard American English (SAE). The benchmark reveals a clear, systematic bias: on average, the model scores AAE text as 1.8 times more toxic and 8.8 times higher for "identity hate". Second, I introduce an interactive pedagogical tool that makes these abstract biases tangible. The tool's core mechanic, a user-controlled "sensitivity threshold," demonstrates that the biased score itself is not the only harm; instead, the more-concerning harm is the human-set, seemingly neutral policy that ultimately operationalises discrimination. This work provides both statistical evidence of disparate impact and a public-facing tool designed to foster critical AI literacy.

翻译：摘要：随着人工智能驱动的内容审核已渗透日常生活，我们常听到“这AI有偏见”的评价。尽管这常以戏谑口吻说出，这种轻松评论实则反映了深层忧虑：如何确定被标记为“不妥”的网络帖子，并非仅仅是算法偏见的牺牲品？本文采用双重方法研究此问题。首先，我对广泛应用的毒性模型（unitary/toxic-bert）进行定量基准测试，衡量其在非裔美国人英语（AAE）与标准美式英语（SAE）文本间的性能差异。基准测试揭示了清晰且系统性的偏见：平均而言，模型将AAE文本的毒性得分判定为SAE文本的1.8倍，而“身份仇恨”得分则高达8.8倍。其次，我引入一个互动教学工具，使这些抽象偏见变得可感知。该工具的核心机制——用户可控的“敏感度阈值”——表明，有偏见的分数本身并非唯一危害；更具警示性的危害在于人为设定且看似中立的政策，这些政策最终将歧视制度化了。本研究既提供了差异性影响的统计证据，也打造了一个旨在培养批判性人工智能素养的公共工具。