Accurate clinical prognosis requires synthesizing structured Electronic Health Records (EHRs) with real-time physiological signals like the Electrocardiogram (ECG). Large Language Models (LLMs) offer a powerful reasoning engine for this task but struggle to natively process these heterogeneous, non-textual data types. To address this, we propose UniPACT (Unified Prognostic Question Answering for Clinical Time-series), a unified framework for prognostic question answering that bridges this modality gap. UniPACT's core contribution is a structured prompting mechanism that converts numerical EHR data into semantically rich text. This textualized patient context is then fused with representations learned directly from raw ECG waveforms, enabling an LLM to reason over both modalities holistically. We evaluate UniPACT on the comprehensive MDS-ED benchmark, it achieves a state-of-the-art mean AUROC of 89.37% across a diverse set of prognostic tasks including diagnosis, deterioration, ICU admission, and mortality, outperforming specialized baselines. Further analysis demonstrates that our multimodal, multi-task approach is critical for performance and provides robustness in missing data scenarios.

翻译：准确的临床预后需要综合结构化电子健康记录（EHR）与心电图（ECG）等实时生理信号。大型语言模型（LLMs）为此任务提供了强大的推理引擎，但其本身难以处理这些异构的非文本数据类型。为解决这一问题，我们提出了UniPACT（面向临床时间序列的统一预后问答框架），这是一个用于预后问答的统一框架，旨在弥合这种模态鸿沟。UniPACT的核心贡献是一种结构化提示机制，可将数值型EHR数据转换为语义丰富的文本。随后，这种文本化的患者上下文信息与直接从原始ECG波形学习到的表征进行融合，使得LLM能够对两种模态进行整体推理。我们在全面的MDS-ED基准测试上评估UniPACT，其在包括诊断、病情恶化、ICU入住和死亡率在内的多种预后任务中，实现了最先进的平均AUROC 89.37%，优于专门的基线模型。进一步的分析表明，我们的多模态、多任务方法对性能提升至关重要，并在数据缺失场景下提供了鲁棒性。