Understanding how high-level concepts are represented within artificial neural networks is a fundamental challenge in the field of artificial intelligence. While existing literature in explainable AI emphasizes the importance of labeling neurons with concepts to understand their functioning, they mostly focus on identifying what stimulus activates a neuron in most cases, this corresponds to the notion of recall in information retrieval. We argue that this is only the first-part of a two-part job, it is imperative to also investigate neuron responses to other stimuli, i.e., their precision. We call this the neuron labels error margin.
翻译:理解人工神经网络中高层概念的表示方式是人工智能领域的一项根本性挑战。尽管可解释人工智能领域的现有文献强调了用概念标记神经元以理解其功能的重要性,但大多数研究主要关注识别何种刺激能激活神经元——这在大部分情况下对应于信息检索中的“召回率”概念。我们认为这只是两步任务中的第一步,还必须探究神经元对其他刺激的响应,即其“精确率”。我们称之为神经元标签的误差裕度。