Evaluating the quality of slide-based multimedia instruction is challenging. Existing methods like manual assessment, reference-based metrics, and large language model evaluators face limitations in scalability, context capture, or bias. In this paper, we introduce LecEval, an automated metric grounded in Mayer's Cognitive Theory of Multimedia Learning, to evaluate multimodal knowledge acquisition in slide-based learning. LecEval assesses effectiveness using four rubrics: Content Relevance (CR), Expressive Clarity (EC), Logical Structure (LS), and Audience Engagement (AE). We curate a large-scale dataset of over 2,000 slides from more than 50 online course videos, annotated with fine-grained human ratings across these rubrics. A model trained on this dataset demonstrates superior accuracy and adaptability compared to existing metrics, bridging the gap between automated and human assessments. We release our dataset and toolkits at https://github.com/JoylimJY/LecEval.

翻译：评估基于幻灯片的多媒体教学质量具有挑战性。现有方法如人工评估、基于参考的指标以及大型语言模型评估器，在可扩展性、上下文捕捉或偏差方面均存在局限。本文提出LecEval，一种基于Mayer多媒体学习认知理论的自动化指标，用于评估基于幻灯片学习中的多模态知识获取。LecEval通过四个维度评估教学效果：内容相关性、表达清晰度、逻辑结构以及受众参与度。我们构建了一个大规模数据集，包含来自50多个在线课程视频的2000多张幻灯片，并标注了这些维度上细粒度的人工评分。基于此数据集训练的模型，相较于现有指标，展现出更优的准确性和适应性，弥合了自动化评估与人工评估之间的差距。我们的数据集和工具包发布于 https://github.com/JoylimJY/LecEval。