Predicting human gaze is important in Human-Computer Interaction (HCI). However, to practically serve HCI applications, gaze prediction models must be scalable, fast, and accurate in their spatial and temporal gaze predictions. Recent scanpath prediction models focus on goal-directed attention (search). Such models are limited in their application due to a common approach relying on trained target detectors for all possible objects, and the availability of human gaze data for their training (both not scalable). In response, we pose a new task called ZeroGaze, a new variant of zero-shot learning where gaze is predicted for never-before-searched objects, and we develop a novel model, Gazeformer, to solve the ZeroGaze problem. In contrast to existing methods using object detector modules, Gazeformer encodes the target using a natural language model, thus leveraging semantic similarities in scanpath prediction. We use a transformer-based encoder-decoder architecture because transformers are particularly useful for generating contextual representations. Gazeformer surpasses other models by a large margin on the ZeroGaze setting. It also outperforms existing target-detection models on standard gaze prediction for both target-present and target-absent search tasks. In addition to its improved performance, Gazeformer is more than five times faster than the state-of-the-art target-present visual search model.

翻译：预测人类注视点在人机交互（HCI）中至关重要。然而，为了切实服务于人机交互应用，注视预测模型必须在空间和时间注视预测上具备可扩展性、快速性和准确性。最近的扫描路径预测模型主要关注目标导向注意力（搜索）。这类模型的应用受到限制，因为它们通常依赖针对所有可能物体的预训练目标检测器，并且需要大量人类注视数据进行训练（两者均缺乏可扩展性）。针对这一问题，我们提出了一项名为ZeroGaze的新任务，这是零样本学习的一个变体，旨在预测从未搜索过的物体的注视点。我们开发了一种新颖的模型Gazeformer来解决ZeroGaze问题。与使用目标检测器模块的现有方法不同，Gazeformer利用自然语言模型对目标进行编码，从而在扫描路径预测中利用语义相似性。我们采用了基于Transformer的编码器-解码器架构，因为Transformer特别擅长生成上下文表示。在ZeroGaze设置中，Gazeformer以显著优势超越了其他模型。在标准注视预测任务中（包括目标存在和目标缺失的搜索），它的表现也优于现有的目标检测模型。除了性能提升外，Gazeformer的速度比最先进的目标存在视觉搜索模型快五倍以上。