Optional type annotations allow for enriching dynamic programming languages with static typing features like better Integrated Development Environment (IDE) support, more precise program analysis, and early detection and prevention of type-related runtime errors. Machine learning-based type inference promises interesting results for automating this task. However, the practical usage of such systems depends on their ability to generalize across different domains, as they are often applied outside their training domain. In this work, we investigate Type4Py as a representative of state-of-the-art deep learning-based type inference systems, by conducting extensive cross-domain experiments. Thereby, we address the following problems: class imbalances, out-of-vocabulary words, dataset shifts, and unknown classes. To perform such experiments, we use the datasets ManyTypes4Py and CrossDomainTypes4Py. The latter we introduce in this paper. Our dataset enables the evaluation of type inference systems in different domains of software projects and has over 1,000,000 type annotations mined on the platforms GitHub and Libraries. It consists of data from the two domains web development and scientific calculation. Through our experiments, we detect that the shifts in the dataset and the long-tailed distribution with many rare and unknown data types decrease the performance of the deep learning-based type inference system drastically. In this context, we test unsupervised domain adaptation methods and fine-tuning to overcome these issues. Moreover, we investigate the impact of out-of-vocabulary words.

翻译：可选类型注解允许为动态编程语言增添静态类型特性，例如更好的集成开发环境（IDE）支持、更精确的程序分析，以及早期检测和预防与类型相关的运行时错误。基于机器学习的类型推断在自动化这一任务中展现出令人期待的结果。然而，此类系统的实际应用取决于其跨领域泛化的能力，因为它们常被应用于训练领域之外。在本工作中，我们以Type4Py作为最先进的深度学习类型推断系统的代表，通过开展广泛的跨领域实验对其进行研究。据此，我们解决了以下问题：类别不平衡、词汇表外词、数据集偏移以及未知类别。为开展此类实验，我们使用了ManyTypes4Py和CrossDomainTypes4Py数据集。后者是本文中引入的数据集。我们的数据集能够评估类型推断系统在不同软件项目领域中的表现，包含从GitHub和Libraries平台上挖掘的超过1,000,000个类型注解。它由网页开发和科学计算两个领域的数据组成。通过实验，我们发现数据集偏移以及包含许多罕见和未知数据类型的重尾分布会显著降低基于深度学习的类型推断系统的性能。在此背景下，我们测试了无监督领域自适应方法和微调技术以克服这些问题。此外，我们还研究了词汇表外词的影响。