End-to-end autonomous driving (E2EAD) systems, which learn to predict future trajectories directly from sensor data, are fundamentally challenged by the inherent spatio-temporal imbalance of trajectory data. This imbalance creates a significant optimization burden, causing models to learn spurious correlations instead of causal inference, while also prioritizing uncertain, distant predictions, thereby compromising immediate safety. To address these issues, we propose ResAD, a novel Normalized Residual Trajectory Modeling framework. Instead of predicting the future trajectory directly, our approach reframes the learning task to predict the residual deviation from a deterministic inertial reference. The inertial reference serves as a counterfactual, forcing the model to move beyond simple pattern recognition and instead identify the underlying causal factors (e.g., traffic rules, obstacles) that necessitate deviations from a default, inertially-guided path. To deal with the optimization imbalance caused by uncertain, long-term horizons, ResAD further incorporates Point-wise Normalization of the predicted residual. It re-weights the optimization objective, preventing large-magnitude errors associated with distant, uncertain waypoints from dominating the learning signal. Extensive experiments validate the effectiveness of our framework. On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy with only two denoising steps, demonstrating that our approach significantly simplifies the learning task and improves model performance. The code will be released to facilitate further research.

翻译：端到端自动驾驶系统直接从传感器数据中学习预测未来轨迹，其根本挑战在于轨迹数据固有的时空不平衡性。这种不平衡性带来了显著的优化负担，导致模型学习虚假相关性而非因果推理，同时优先处理不确定的远距离预测，从而损害即时安全性。为解决这些问题，我们提出ResAD——一种新颖的归一化残差轨迹建模框架。该方法不直接预测未来轨迹，而是将学习任务重构为预测相对于确定性惯性参考的残差偏移量。惯性参考作为反事实基准，迫使模型超越简单的模式识别，转而识别导致偏离默认惯性引导路径的根本因果因素（如交通规则、障碍物）。针对不确定长时域引发的优化不平衡问题，ResAD进一步引入预测残差的逐点归一化机制，通过重新加权优化目标，防止由遥远不确定路径点产生的大幅值误差主导学习信号。大量实验验证了我们框架的有效性：在NAVSIM基准测试中，ResAD使用仅含两步去噪的朴素扩散策略即达到88.6的PDMS最优性能，证明该方法能显著简化学习任务并提升模型性能。代码将开源以促进后续研究。