Shadow removal under diverse lighting conditions requires disentangling illumination from intrinsic reflectance, a challenge compounded when physical priors are not properly aligned. We propose PhaSR (Physically Aligned Shadow Removal), addressing this through dual-level prior alignment to enable robust performance from single-light shadows to multi-source ambient lighting. First, Physically Aligned Normalization (PAN) performs closed-form illumination correction via Gray-world normalization, log-domain Retinex decomposition, and dynamic range recombination, suppressing chromatic bias. Second, Geometric-Semantic Rectification Attention (GSRA) extends differential attention to cross-modal alignment, harmonizing depth-derived geometry with DINO-v2 semantic embeddings to resolve modal conflicts under varying illumination. Experiments show competitive performance in shadow removal with lower complexity and generalization to ambient lighting where traditional methods fail under multi-source illumination. Our source code is available at https://github.com/ming053l/PhaSR.

翻译：在多样化光照条件下的阴影去除需要将光照与固有反射率解耦，当物理先验未正确对齐时，这一问题尤为复杂。我们提出PhaSR（物理对齐阴影去除）方法，通过双层级先验对齐来解决该问题，从而实现从单光源阴影到多源环境光照的鲁棒性能。首先，物理对齐归一化（PAN）通过灰度世界归一化、对数域Retinex分解和动态范围重组进行闭式光照校正，抑制色度偏差。其次，几何语义校正注意力（GSRA）将差分注意力扩展至跨模态对齐，协调深度衍生的几何信息与DINO-v2语义嵌入，以解决变化光照下的模态冲突。实验表明，该方法在阴影去除方面具有竞争力的性能，且复杂度更低，并能泛化至传统方法在多源光照下失效的环境光照场景。我们的源代码公开于https://github.com/ming053l/PhaSR。