Meetings play a critical infrastructural role in the coordination of work. In recent years, due to shift to hybrid and remote work, more meetings are moving to online Computer Mediated Spaces. This has led to new problems (e.g. more time spent in less engaging meetings) and new opportunities (e.g. automated transcription/captioning and recap support). Recent advances in large language models (LLMs) for dialog summarization have the potential to improve the experience of meetings by reducing individuals' meeting load and increasing the clarity and alignment of meeting outputs. Despite this potential, they face technological limitation due to long transcripts and inability to capture diverse recap needs based on user's context. To address these gaps, we design, implement and evaluate in-context a meeting recap system. We first conceptualize two salient recap representations -- important highlights, and a structured, hierarchical minutes view. We develop a system to operationalize the representations with dialogue summarization as its building blocks. Finally, we evaluate the effectiveness of the system with seven users in the context of their work meetings. Our findings show promise in using LLM-based dialogue summarization for meeting recap and the need for both representations in different contexts. However, we find that LLM-based recap still lacks an understanding of whats personally relevant to participants, can miss important details, and mis-attributions can be detrimental to group dynamics. We identify collaboration opportunities such as a shared recap document that a high quality recap enables. We report on implications for designing AI systems to partner with users to learn and improve from natural interactions to overcome the limitations related to personal relevance and summarization quality.
翻译:会议在工作协调中扮演着关键的基础设施角色。近年来,随着混合办公和远程办公的转变,越来越多的会议迁移至在线计算机媒介空间。这带来了新问题(例如,在参与度较低的会议上花费更多时间)和新机遇(例如,自动转录/字幕生成和回顾支持)。基于大型语言模型(LLM)的对话摘要技术的最新进展,有望通过减少个人会议负担并提升会议产出的清晰度和一致性来改善会议体验。然而,尽管具有这种潜力,由于会议记录冗长且无法基于用户情境捕捉多样化的回顾需求,这些技术仍面临局限性。为弥补这些不足,我们设计、实现并在实际场景中评估了一套会议回顾系统。我们首先概念化了两种关键的回顾表示形式——重要亮点和结构化、层次化的会议纪要视图。接着,我们开发了一套系统,以对话摘要为构建模块来具体化这些表示形式。最后,我们结合七位用户的实际工作会议场景,评估了该系统的有效性。研究结果表明,基于LLM的对话摘要技术用于会议回顾具有潜力,且在不同情境下需要同时采用这两种表示形式。然而,我们也发现基于LLM的回顾仍无法理解参与者个人相关的内容,可能遗漏重要细节,并且错误归因可能损害团队动态。我们识别出协作机会,例如高质量回顾所支持的共享回顾文档。我们报告了设计AI系统以与用户合作、从自然交互中学习并改进,从而克服个人相关性和摘要质量局限性的启示。