We propose Unified Model of Saliency and Scanpaths (UMSS) -- a model that learns to predict visual saliency and scanpaths (i.e. sequences of eye fixations) on information visualisations. Although scanpaths provide rich information about the importance of different visualisation elements during the visual exploration process, prior work has been limited to predicting aggregated attention statistics, such as visual saliency. We present in-depth analyses of gaze behaviour for different information visualisation elements (e.g. Title, Label, Data) on the popular MASSVIS dataset. We show that while, overall, gaze patterns are surprisingly consistent across visualisations and viewers, there are also structural differences in gaze dynamics for different elements. Informed by our analyses, UMSS first predicts multi-duration element-level saliency maps, then probabilistically samples scanpaths from them. Extensive experiments on MASSVIS show that our method consistently outperforms state-of-the-art methods with respect to several, widely used scanpath and saliency evaluation metrics. Our method achieves a relative improvement in sequence score of 11.5% for scanpath prediction, and a relative improvement in Pearson correlation coefficient of up to 23.6% for saliency prediction. These results are auspicious and point towards richer user models and simulations of visual attention on visualisations without the need for any eye tracking equipment.

翻译：我们提出统一显著性及视线路径模型（UMSS）——一种学习预测信息可视化中视觉显著性与视线路径（即眼动注视点序列）的模型。尽管视线路径能提供视觉探索过程中不同可视化元素重要性的丰富信息，但先前研究仅限于预测聚合注意力统计量（如视觉显著性）。我们在流行的MASSVIS数据集上，针对不同信息可视化元素（如标题、标签、数据）的凝视行为进行了深入分析。研究表明，尽管整体上不同可视化及其观看者的凝视模式惊人地一致，但不同元素的凝视动态也存在结构性差异。基于分析结果，UMSS首先预测多时长元素级显著性图，然后从中概率性地采样视线路径。在MASSVIS上的大量实验表明，该方法在多项广泛使用的视线路径与显著性评估指标上持续优于现有最优方法。在视线路径预测的序列评分上获得11.5%的相对提升，在显著性预测的皮尔逊相关系数上最高获得23.6%的相对提升。这些成果令人鼓舞，预示着无需任何眼动追踪设备即可构建更丰富的注意力用户模型与可视化注视行为模拟。