Trajectory prediction plays a vital role in understanding pedestrian movement for applications such as autonomous driving and robotics. Current trajectory prediction models depend on long, complete, and accurately observed sequences from visual modalities. Nevertheless, real-world situations often involve obstructed cameras, missed objects, or objects out of sight due to environmental factors, leading to incomplete or noisy trajectories. To overcome these limitations, we propose LTrajDiff, a novel approach that treats objects obstructed or out of sight as equally important as those with fully visible trajectories. LTrajDiff utilizes sensor data from mobile phones to surmount out-of-sight constraints, albeit introducing new challenges such as modality fusion, noisy data, and the absence of spatial layout and object size information. We employ a denoising diffusion model to predict precise layout sequences from noisy mobile data using a coarse-to-fine diffusion strategy, incorporating the RMS, Siamese Masked Encoding Module, and MFM. Our model predicts layout sequences by implicitly inferring object size and projection status from a single reference timestamp or significantly obstructed sequences. Achieving SOTA results in randomly obstructed experiments and extremely short input experiments, our model illustrates the effectiveness of leveraging noisy mobile data. In summary, our approach offers a promising solution to the challenges faced by layout sequence and trajectory prediction models in real-world settings, paving the way for utilizing sensor data from mobile phones to accurately predict pedestrian bounding box trajectories. To the best of our knowledge, this is the first work that addresses severely obstructed and extremely short layout sequences by combining vision with noisy mobile modality, making it the pioneering work in the field of layout sequence trajectory prediction.

翻译：轨迹预测在理解行人移动方面发挥着关键作用，应用于自动驾驶和机器人等领域。当前的轨迹预测模型依赖于视觉模态中长序列、完整且准确观测的数据。然而，现实场景中常因环境因素导致摄像头遮挡、目标遗漏或目标超出视野，从而产生不完整或带噪声的轨迹。为克服这些局限，我们提出LTrajDiff，一种将遮挡或超出视野的目标与完全可见轨迹的目标同等对待的新方法。LTrajDiff利用手机传感器数据突破视野限制，尽管这引入了模态融合、噪声数据以及缺乏空间布局和物体尺寸信息等新挑战。我们采用去噪扩散模型，通过由粗到细的扩散策略，结合RMS、孪生掩码编码模块和MFM，从噪声移动数据中预测精确的布局序列。我们的模型通过隐式推断单个参考时间戳或严重遮挡序列中的物体尺寸和投影状态来预测布局序列。在随机遮挡实验和极短输入实验中达到最优结果，验证了利用噪声移动数据的有效性。总之，本方法为布局序列和轨迹预测模型在现实场景中面临的挑战提供了有前景的解决方案，为利用手机传感器数据精确预测行人边界框轨迹铺平了道路。据我们所知，这是首个通过融合视觉与噪声移动模态解决严重遮挡和极短布局序列的工作，开创了布局序列轨迹预测领域的新方向。