Time series data in real-world applications such as healthcare, climate modeling, and finance are often irregular, multimodal, and messy, with varying sampling rates, asynchronous modalities, and pervasive missingness. However, existing benchmarks typically assume clean, regularly sampled, unimodal data, creating a significant gap between research and real-world deployment. We introduce Time-IMM, a dataset specifically designed to capture cause-driven irregularity in multimodal multivariate time series. Time-IMM represents nine distinct types of time series irregularity, categorized into trigger-based, constraint-based, and artifact-based mechanisms. Complementing the dataset, we introduce IMM-TSF, a benchmark library for forecasting on irregular multimodal time series, enabling asynchronous integration and realistic evaluation. IMM-TSF includes specialized fusion modules, including a timestamp-to-text fusion module and a multimodality fusion module, which support both recency-aware averaging and attention-based integration strategies. Empirical results demonstrate that explicitly modeling multimodality on irregular time series data leads to substantial gains in forecasting performance. Time-IMM and IMM-TSF provide a foundation for advancing time series analysis under real-world conditions. The dataset is publicly available at https://github.com/blacksnail789521/Time-IMM, and the benchmark library can be accessed at https://github.com/blacksnail789521/IMM-TSF. Project page: https://blacksnail789521.github.io/time-imm-project-page/

翻译：在医疗保健、气候建模和金融等实际应用中，时间序列数据往往具有不规则性、多模态性和杂乱性，表现为采样率各异、模态异步以及普遍存在的数据缺失。然而，现有基准测试通常假设数据是干净、规则采样且单模态的，这导致研究与实际部署之间存在显著差距。本文介绍了Time-IMM，这是一个专门为捕捉多模态多元时间序列中由原因驱动的不规则性而设计的数据集。Time-IMM涵盖了九种不同类型的时间序列不规则性，并将其归类为基于触发、基于约束和基于伪影的机制。作为数据集的补充，我们提出了IMM-TSF，一个用于不规则多模态时间序列预测的基准测试库，支持异步集成与真实场景评估。IMM-TSF包含专门的融合模块，包括时间戳到文本融合模块和多模态融合模块，这些模块同时支持基于时效性的平均策略和基于注意力的集成策略。实验结果表明，在不规则时间序列数据上显式建模多模态特性能显著提升预测性能。Time-IMM与IMM-TSF为推进真实场景下的时间序列分析奠定了基础。数据集已公开于https://github.com/blacksnail789521/Time-IMM，基准测试库可通过https://github.com/blacksnail789521/IMM-TSF访问。项目页面：https://blacksnail789521.github.io/time-imm-project-page/