The automation of a large number of software engineering tasks is becoming possible thanks to Machine Learning (ML). Central to applying ML to software artifacts (like source or executable code) is converting them into forms suitable for learning. Traditionally, researchers have relied on manually selected features, based on expert knowledge which is sometimes imprecise and generally incomplete. Representation learning has allowed ML to automatically choose suitable representations and relevant features. Yet, for Android-related tasks, existing models like apk2vec focus on whole-app levels, or target specific tasks like smali2vec, which limits their applicability. Our work is part of a new line of research that investigates effective, task-agnostic, and fine-grained universal representations of bytecode to mitigate both of these two limitations. Such representations aim to capture information relevant to various low-level downstream tasks (e.g., at the class-level). We are inspired by the field of Natural Language Processing, where the problem of universal representation was addressed by building Universal Language Models, such as BERT, whose goal is to capture abstract semantic information about sentences, in a way that is reusable for a variety of tasks. We propose DexBERT, a BERT-like Language Model dedicated to representing chunks of DEX bytecode, the main binary format used in Android applications. We empirically assess whether DexBERT is able to model the DEX language and evaluate the suitability of our model in three distinct class-level software engineering tasks: Malicious Code Localization, Defect Prediction, and Component Type Classification. We also experiment with strategies to deal with the problem of catering to apps having vastly different sizes, and we demonstrate one example of using our technique to investigate what information is relevant to a given task.

翻译：得益于机器学习（ML），大量软件工程任务的自动化正成为可能。将机器学习应用于软件制品（如源代码或可执行代码）的核心是将它们转化为适合学习的形式。传统上，研究者依赖基于专家知识手动选择的特征，但这种方式有时不够精确且通常不完整。表示学习使得机器学习能够自动选择适当的表示和相关特征。然而，对于安卓相关任务，现有模型如apk2vec聚焦于整体应用级别，或目标特定任务如smali2vec，这限制了它们的适用性。我们的工作属于一条新的研究路线，旨在探索高效、任务无关且细粒度的字节码通用表示，以缓解这两个局限性。此类表示旨在捕获与各种低级下游任务（例如，类级别任务）相关的信息。我们受自然语言处理领域的启发，其中通用表示问题通过构建通用语言模型（如BERT）来解决，这些模型的目标是捕获句子的抽象语义信息，使其可重用于多种任务。我们提出了DexBERT，一种类似BERT的语言模型，专门用于表示DEX字节码（安卓应用中使用的主要二进制格式）的片段。我们通过实验评估DexBERT是否能建模DEX语言，并评估我们的模型在三个不同的类级别软件工程任务中的适用性：恶意代码定位、缺陷预测和组件类型分类。我们还尝试了策略以应对适配大小差异巨大的应用程序的问题，并演示了如何使用我们的技术来研究哪些信息与给定任务相关的一个示例。