Understanding generalization is crucial to confidently engineer and deploy machine learning models, especially when deployment implies a shift in the data domain. For such domain adaptation problems, we seek generalization bounds which are tractably computable and tight. If these desiderata can be reached, the bounds can serve as guarantees for adequate performance in deployment. However, in applications where deep neural networks are the models of choice, deriving results which fulfill these remains an unresolved challenge; most existing bounds are either vacuous or has non-estimable terms, even in favorable conditions. In this work, we evaluate existing bounds from the literature with potential to satisfy our desiderata on domain adaptation image classification tasks, where deep neural networks are preferred. We find that all bounds are vacuous and that sample generalization terms account for much of the observed looseness, especially when these terms interact with measures of domain shift. To overcome this and arrive at the tightest possible results, we combine each bound with recent data-dependent PAC-Bayes analysis, greatly improving the guarantees. We find that, when domain overlap can be assumed, a simple importance weighting extension of previous work provides the tightest estimable bound. Finally, we study which terms dominate the bounds and identify possible directions for further improvement.

翻译：理解泛化对于可靠设计和部署机器学习模型至关重要，尤其是在部署涉及数据域迁移时。针对此类域适应问题，我们寻求可计算且紧致的泛化界。若这些期望属性得以实现，该界可作为部署中性能充分性的保证。然而，在以深度神经网络为首选模型的应用中，推导满足这些条件的结论仍是一项未解决的挑战：即便在有利条件下，现有界大多要么是空洞的，要么包含不可估计项。本研究评估了文献中可能满足我们期望的域适应图像分类任务界（深度神经网络在此类任务中具有优势）。我们发现所有界均存在空洞性，且样本泛化项是造成观测松驰的主要因素，尤其当这些项与域迁移度量相互作用时。为克服此问题并获得尽可能紧致的结果，我们将每个界与近期基于数据的PAC-贝叶斯分析相结合，显著改进了保证质量。研究发现，在可假设域重叠的情况下，先前工作的简单重要性加权扩展提供了最紧致的可估计界。最后，我们研究了主导各项的边界因素，并识别出进一步优化的可能方向。