Information workers' productivity is significantly influenced by their cognitive states and physiological responses. AI assistants such as ChatGPT, Copilot, and others have become integral components of knowledge-intensive workplaces. These AI assistants utilize pre-defined user preferences and chat interaction histories, thus confining themselves to reactive exchanges, lacking sufficient adaptability. Consequently, they fail to cater to individual user preferences and are unable to adapt to their psychophysiological states, diminishing potential productivity gains. To bridge this gap, we introduce AwareLLM, a novel multimodal framework that integrates egocentric vision, pupillometry, eye-gaze tracking, posture detection, heart activity, and the inferencing capabilities of large language models (LLMs) to create a proactive and context-aware ecosystem. AwareLLM dynamically adapts to users' psychophysiological states while analyzing temporal patterns and behavioral tendencies to provide personalized and timely interventions. We evaluated AwareLLM through a user study with 20 participants, comparing it to a standard LLM assistant across multiple tasks. Our results show statistically significant improvements in task performance, along with reductions in cognitive fatigue and mental demand. Participants described AwareLLM's personalized interventions as timely and relevant, helping them boost their confidence and deepen engagement with their work. AwareLLM opens new avenues for Human-AI collaboration where technology adapts to our needs rather than us adhering to technological constraints.

翻译：信息工作者的生产力显著受到其认知状态和生理反应的影响。AI助手（如ChatGPT、Copilot等）已成为知识密集型工作场所不可或缺的组成部分。这些AI助手依赖预设的用户偏好和聊天交互历史，因而局限于被动式应答，缺乏足够的适应性。因此，它们无法满足个体用户的偏好，也无法适应其心理生理状态，从而削弱了潜在的生产力提升。为弥补这一差距，我们提出了AwareLLM——一种新颖的多模态框架，该框架整合了自我中心视觉、瞳孔测量、眼动追踪、姿势检测、心脏活动以及大语言模型的推理能力，以构建一个主动且具备上下文感知能力的生态系统。AwareLLM在动态适应用户心理生理状态的同时，分析时间模式和行为倾向，以提供个性化且及时的干预。我们通过一项包含20名参与者的用户研究对AwareLLM进行了评估，并将其在多项任务中与标准LLM助手进行了对比。实验结果显示，任务表现有统计学意义上的显著提升，同时认知疲劳和脑力需求有所降低。参与者认为AwareLLM的个性化干预及时且相关，有助于增强他们的信心并加深对工作的投入度。AwareLLM为人机协作开辟了新途径——使技术主动适应人类需求，而非让人屈从于技术约束。