Hallucination is an apparent perception in the absence of real external sensory stimuli. An auditory hallucination is a perception of hearing sounds that are not real. A common form of auditory hallucination is hearing voices in the absence of any speakers which is known as Auditory Verbal Hallucination (AVH). AVH is fragments of the mind's creation that mostly occur in people diagnosed with mental illnesses such as bipolar disorder and schizophrenia. Assessing the valence of hallucinated voices (i.e., how negative or positive voices are) can help measure the severity of a mental illness. We study N=435 individuals, who experience hearing voices, to assess auditory verbal hallucination. Participants report the valence of voices they hear four times a day for a month through ecological momentary assessments with questions that have four answering scales from ``not at all'' to ``extremely''. We collect these self-reports as the valence supervision of AVH events via a mobile application. Using the application, participants also record audio diaries to describe the content of hallucinated voices verbally. In addition, we passively collect mobile sensing data as contextual signals. We then experiment with how predictive these linguistic and contextual cues from the audio diary and mobile sensing data are of an auditory verbal hallucination event. Finally, using transfer learning and data fusion techniques, we train a neural net model that predicts the valance of AVH with a performance of 54\% top-1 and 72\% top-2 F1 score.

翻译：幻听是在缺乏真实外部 sensory 刺激的情况下产生的知觉。言语性幻听是指听到不真实声音的知觉，其中一种常见形式是在没有说话者的情况下听到声音，即言语性幻听（AVH）。AVH 是大脑创造出的片段，主要发生在被诊断患有双相情感障碍、精神分裂症等精神疾病的人群中。评估幻听声音的效价（即声音的负面或正面程度）有助于衡量精神疾病的严重程度。本研究对 435 名经历幻听的个体进行了言语性幻听评估。参与者通过生态瞬时评估，每天四次报告一个月内所听到声音的效价，评估问题包括四个回答等级，从“完全没有”到“极其”。我们通过移动应用程序收集这些自我报告作为 AVH 事件的效价监督。参与者还通过该应用程序录制音频日记，口头描述幻听声音的内容。此外，我们被动收集移动传感数据作为背景信号。随后，我们实验了这些来自音频日记和移动传感数据的语言和背景线索对言语性幻听事件的预测能力。最后，利用迁移学习和数据融合技术，我们训练了一个神经网络模型，该模型预测 AVH 的效价，实现了 54% 的 top-1 和 72% 的 top-2 F1 得分。