To accelerate the inference of heavy Multimodal Large Language Models (MLLMs), this study rethinks the current landscape of training-free token reduction research. We regret to find that the critical components of existing methods are tightly intertwined, with their interconnections and effects remaining unclear for comparison, transfer, and expansion. Therefore, we propose a unified ''filter-correlate-compress'' paradigm that decomposes the token reduction into three distinct stages within a pipeline, maintaining consistent design objectives and elements while allowing for unique implementations. We additionally demystify the popular works and subsume them into our paradigm to showcase its universality. Finally, we offer a suite of methods grounded in the paradigm, striking a balance between speed and accuracy throughout different phases of the inference. Experimental results across 10 benchmarks indicate that our methods can achieve up to an 82.4% reduction in FLOPs with a minimal impact on performance, simultaneously surpassing state-of-the-art training-free methods. Our project page is at https://ficoco-accelerate.github.io/.

翻译：为加速重型多模态大语言模型（MLLMs）的推理，本研究重新审视了当前无需训练的令牌缩减研究现状。我们遗憾地发现，现有方法的核心组件紧密交织，其相互关联与影响在比较、迁移和扩展方面仍不明确。因此，我们提出了一个统一的“过滤-关联-压缩”范式，将令牌缩减分解为流水线中三个独立的阶段，在保持设计目标与要素一致的同时，允许独特的实现方式。我们进一步解析了现有主流工作，并将其纳入我们的范式以展示其普适性。最后，我们基于该范式提出了一系列方法，在推理的不同阶段实现了速度与精度之间的平衡。在10个基准测试上的实验结果表明，我们的方法能以极小的性能损失实现高达82.4%的FLOPs缩减，同时超越了当前最先进的无需训练方法。项目页面位于 https://ficoco-accelerate.github.io/。