Personality traits influence how individuals engage, behave, and make decisions during the information-seeking process. However, few studies have linked personality to observable search behaviors. This study aims to characterize personality traits through a multimodal time-series model that integrates eye-tracking data and gaze missingness-periods when the user's gaze is not captured. This approach is based on the idea that people often look away when they think, signaling disengagement or reflection. We conducted a user study with 25 participants, who used an interactive application on an iPad, allowing them to engage with digital artifacts from a museum. We rely on raw gaze data from an eye tracker, minimizing preprocessing so that behavioral patterns can be preserved without substantial data cleaning. From this perspective, we trained models to predict personality traits using gaze signals. Our results from a five-fold cross-validation study demonstrate strong predictive performance across all five dimensions: Neuroticism (Macro F1 = 77.69%), Conscientiousness (74.52%), Openness (77.52%), Agreeableness (73.09%), and Extraversion (76.69%). The ablation study examines whether the absence of gaze information affects the model performance, demonstrating that incorporating missingness improves multimodal time-series modeling. The full model, which integrates both time-series signals and missingness information, achieves 10-15% higher accuracy and macro F1 scores across all Big Five traits compared to the model without time-series signals and missingness. These findings provide evidence that personality can be inferred from search-related gaze behavior and demonstrate the value of incorporating missing gaze data into time-series multimodal modeling.
翻译:人格特质影响个体在信息寻求过程中的参与方式、行为表现与决策制定。然而,将人格特质与可观测的搜索行为相关联的研究尚不多见。本研究旨在通过一种多模态时间序列模型来刻画人格特质,该模型整合了眼动追踪数据以及用户注视未被捕获时的注视缺失时段。该方法基于以下观点:人们在思考时常会移开视线,这标志着脱离或反思状态。我们开展了一项用户研究,共有25名参与者使用iPad上的交互式应用程序与博物馆数字藏品进行互动。我们基于眼动仪采集的原始注视数据,尽可能减少预处理步骤,以保留行为模式而无需大量数据清洗。基于此视角,我们训练了利用注视信号预测人格特质的模型。通过五折交叉验证实验,我们的结果显示模型在所有五个人格维度上均表现出较强的预测性能:神经质(宏观F1 = 77.69%)、尽责性(74.52%)、开放性(77.52%)、宜人性(73.09%)和外向性(76.69%)。消融实验检验了注视信息缺失是否影响模型性能,结果表明纳入缺失信息能提升多模态时间序列建模效果。完整模型同时整合时间序列信号与缺失信息,相较于未包含时间序列信号与缺失信息的模型,其在大五人格所有特质上的准确率与宏观F1分数均提升10-15%。这些发现为通过搜索相关注视行为推断人格提供了证据,并证明了将缺失注视数据纳入时间序列多模态建模的价值。