The aim of video summarization is to shorten videos automatically while retaining the key information necessary to convey the overall story. Video summarization methods mainly rely on visual factors, such as visual consecutiveness and diversity, which may not be sufficient to fully understand the content of the video. There are other non-visual factors, such as interestingness, representativeness, and storyline consistency that should also be considered for generating high-quality video summaries. Current methods do not adequately take into account these non-visual factors, resulting in suboptimal performance. In this work, a new approach to video summarization is proposed based on insights gained from how humans create ground truth video summaries. The method utilizes a conditional modeling perspective and introduces multiple meaningful random variables and joint distributions to characterize the key components of video summarization. Helper distributions are employed to improve the training of the model. A conditional attention module is designed to mitigate potential performance degradation in the presence of multi-modal input. The proposed video summarization method incorporates the above innovative design choices that aim to narrow the gap between human-generated and machine-generated video summaries. Extensive experiments show that the proposed approach outperforms existing methods and achieves state-of-the-art performance on commonly used video summarization datasets.

翻译：视频摘要的目标是自动缩短视频时长，同时保留传达整体故事所需的关键信息。现有视频摘要方法主要依赖视觉因素（如视觉连续性和多样性），但仅凭这些可能难以充分理解视频内容。除视觉因素外，生成高质量视频摘要还应考虑趣味性、代表性及故事情节一致性等其他非视觉因素。当前方法未能充分纳入这些非视觉因素，导致性能欠佳。本研究基于人类构建真实视频摘要的认知机制，提出一种新的视频摘要方法。该方法采用条件建模视角，引入多个有意义的随机变量及联合分布来表征视频摘要的关键组件，并利用辅助分布改进模型训练。针对多模态输入可能导致的性能退化问题，设计了条件注意力模块。所提出的视频摘要方法融合上述创新设计，旨在缩小人工生成与机器生成视频摘要之间的差距。大量实验表明，该方法在常用视频摘要数据集上超越现有方法，达到当前最优性能。