Dense 3D reconstruction from continuous image streams requires both accurate geometric aggregation and stable long-term memory management. Recent feed-forward reconstruction frameworks integrate observations through persistent memory representations, yet most rely primarily on appearance-based similarity when updating memory. Such appearance-driven integration often leads to redundant accumulation of observations and unstable geometry when viewpoint changes occur. In this work, we propose a ray-aware pointer memory for streaming 3D reconstruction that explicitly models both spatial location and viewing direction within a unified memory representation. Each memory pointer stores its 3D position, associated ray direction, and feature embedding, allowing the system to reason jointly about geometric proximity and viewpoint consistency. Based on this representation, we introduce an adaptive pointer update strategy that replaces traditional fusion-based memory compression with a retain-or-replace mechanism. Instead of averaging nearby observations, the system selectively retains informative pointers while discarding redundant ones, preserving distinctive geometric structures while maintaining bounded memory growth. Furthermore, the joint reasoning over spatial distance and ray-direction discrepancy enables the system to distinguish between local redundancy, novel observations, and potential loop revisits in a unified manner. When loop candidates are detected, pose refinement is triggered to enforce global geometric consistency across the reconstruction. Extensive experiments demonstrate that the proposed ray-aware memory design significantly improves long-term reconstruction stability and camera pose accuracy while maintaining efficient streaming inference. Our approach provides a principled framework for scalable and drift-resistant online 3D reconstruction from image streams.

翻译：从不间断图像流中进行稠密三维重建，需要同时具备精确的几何聚合和稳定的长期记忆管理能力。近期前馈式重建框架通过持久化记忆表示来整合观测信息，但大多数方法在更新记忆时主要依赖于基于外观的相似性。这种外观驱动的整合方式往往导致观测信息的冗余积累，并在视角变化时引发几何结构不稳定。本文提出一种面向流式三维重建的射线感知指针存储器，该存储器在统一的记忆表示中显式建模了空间位置与观察方向。每个记忆指针存储其三维坐标、关联射线方向及特征嵌入，使系统能够联合推理几何邻近性与视角一致性。基于该表示，我们引入一种自适应指针更新策略，将传统的融合式记忆压缩替换为"保留或替换"机制。系统不再对邻近观测进行平均化处理，而是选择性保留信息量丰富的指针并丢弃冗余指针，在保持边界内存增长的同时保留独特的几何结构。此外，通过联合推理空间距离与射线方向差异，系统能够以统一方式区分局部冗余、新观测信息及潜在的回环访问。当检测到回环候选时，会触发位姿精化以强制重建过程中全局几何一致性。大量实验证明，所提出的射线感知记忆设计在保持高效流式推理的同时，显著提升了长期重建稳定性与相机位姿精度。我们的方法为基于图像流的可扩展、抗漂移在线三维重建提供了原则性框架。