How can we reliably transfer affect models trained in controlled laboratory conditions (in-vitro) to uncontrolled real-world settings (in-vivo)? The information gap between in-vitro and in-vivo applications defines a core challenge of affective computing. This gap is caused by limitations related to affect sensing including intrusiveness, hardware malfunctions and availability of sensors. As a response to these limitations, we introduce the concept of privileged information for operating affect models in real-world scenarios (in the wild). Privileged information enables affect models to be trained across multiple modalities available in a lab, and ignore, without significant performance drops, those modalities that are not available when they operate in the wild. Our approach is tested in two multimodal affect databases one of which is designed for testing models of affect in the wild. By training our affect models using all modalities and then using solely raw footage frames for testing the models, we reach the performance of models that fuse all available modalities for both training and testing. The results are robust across both classification and regression affect modeling tasks which are dominant paradigms in affective computing. Our findings make a decisive step towards realizing affect interaction in the wild.

翻译：如何可靠地将受控实验室条件下（体外）训练的情感模型迁移到非受控真实环境（体内）？体外与体内应用之间的信息鸿沟是情感计算的核心挑战。这一鸿沟源于情感感知的局限性，包括侵入性、硬件故障及传感器可用性。针对这些局限，我们提出了特权信息的概念，用于在真实场景（野外）中运行情感模型。特权信息使情感模型能够利用实验室中可用的多模态数据进行训练，并在野外运行时忽略那些不可用的模态，且性能无显著下降。我们的方法在两个多模态情感数据库上进行了测试，其中一个专门用于野外情感模型评估。通过使用所有模态训练情感模型，再仅用原始视频帧进行模型测试，我们达到了与融合所有模态进行训练和测试的模型相当的性能。该结果在情感计算的核心范式——分类与回归情感建模任务中均表现稳健。我们的发现为实现在真实环境中的情感交互迈出了决定性的一步。