Depth from Defocus (DfD) is the task of estimating a dense metric depth map from a focus stack. Unlike previous works overfitting to a certain dataset, this paper focuses on the challenging and practical setting of zero-shot generalization. We first propose a new real-world DfD benchmark ZEDD, which contains 8.3x more scenes and significantly higher quality images and ground-truth depth maps compared to previous benchmarks. We also design a novel network architecture named FOSSA. FOSSA is a Transformer-based architecture with novel designs tailored to the DfD task. The key contribution is a stack attention layer with a focus distance embedding, allowing efficient information exchange across the focus stack. Finally, we develop a new training data pipeline allowing us to utilize existing large-scale RGBD datasets to generate synthetic focus stacks. Experiment results on ZEDD and other benchmarks show a significant improvement over the baselines, reducing errors by up to 55.7%. The ZEDD benchmark is released at https://zedd.cs.princeton.edu. The code and checkpoints are released at https://github.com/princeton-vl/FOSSA.

翻译：离焦深度估计（DfD）旨在从焦点堆栈中估计密集度量深度图。与先前在特定数据集上过拟合的工作不同，本文聚焦于零样本泛化这一具有挑战性且实用的场景。我们首先提出了一个新的真实世界DfD基准ZEDD，与先前基准相比，其场景数量增加了8.3倍，且图像和真实深度图的质量显著更高。我们还设计了一种名为FOSSA的新型网络架构。FOSSA是基于Transformer的架构，针对DfD任务设计了新颖的结构。关键贡献在于引入带有焦点距离嵌入的堆栈注意力层，使得焦点堆栈间的信息交换更加高效。最后，我们开发了一种新的训练数据流水线，能够利用现有的大规模RGBD数据集生成合成焦点堆栈。在ZEDD及其他基准上的实验结果表明，相较于基线方法，误差最多降低了55.7%。ZEDD基准已在https://zedd.cs.princeton.edu发布。代码和模型权重已公开于https://github.com/princeton-vl/FOSSA。