We depend on our own memory to encode, store, and retrieve our experiences. However, memory lapses can occur. One promising avenue for achieving memory augmentation is through the use of augmented reality head-mounted displays to capture and preserve egocentric videos, a practice commonly referred to as lifelogging. However, a significant challenge arises from the sheer volume of video data generated through lifelogging, as the current technology lacks the capability to encode and store such large amounts of data efficiently. Further, retrieving specific information from extensive video archives requires substantial computational power, further complicating the task of quickly accessing desired content. To address these challenges, we propose a memory augmentation agent that involves leveraging natural language encoding for video data and storing them in a vector database. This approach harnesses the power of large vision language models to perform the language encoding process. Additionally, we propose using large language models to facilitate natural language querying. Our agent underwent extensive evaluation using the QA-Ego4D dataset and achieved state-of-the-art results with a BLEU score of 8.3, outperforming conventional machine learning models that scored between 3.4 and 5.8. Additionally, we conducted a user study in which participants interacted with the human memory augmentation agent through episodic memory and open-ended questions. The results of this study show that the agent results in significantly better recall performance on episodic memory tasks compared to human participants. The results also highlight the agent's practical applicability and user acceptance.

翻译：我们依赖自身记忆对经历进行编码、存储与检索，但记忆疏漏时有发生。利用增强现实头戴设备捕获并保存自我中心视频（通常称为生活日志）是实现记忆增强的一条可行路径。然而，生活日志产生的海量视频数据带来了重大挑战：当前技术缺乏高效编码与存储如此大规模数据的能力。此外，从庞大的视频档案中检索特定信息需要巨大的计算资源，这进一步增加了快速获取目标内容的难度。为应对这些挑战，我们提出一种记忆增强智能体，通过自然语言编码处理视频数据并将其存储于向量数据库。该方法利用大规模视觉语言模型实现语言编码过程，并借助大语言模型实现自然语言查询功能。我们使用QA-Ego4D数据集对该智能体进行了全面评估，其以8.3的BLEU分数取得了最先进的性能表现，显著优于得分在3.4至5.8之间的传统机器学习模型。此外，我们开展了用户研究，参与者通过情景记忆和开放式问题与人类记忆增强智能体进行交互。研究结果表明，在情景记忆任务中，智能体辅助的回忆表现显著优于人类参与者。该结果同时凸显了智能体的实际应用价值与用户接受度。