Code retrieval, which retrieves code snippets based on users' natural language descriptions, is widely used by developers and plays a pivotal role in real-world software development. The advent of deep learning has shifted the retrieval paradigm from lexical-based matching towards leveraging deep learning models to encode source code and queries into vector representations, facilitating code retrieval according to vector similarity. Despite the effectiveness of these models, managing large-scale code database presents significant challenges. Previous research proposes deep hashing-based methods, which generate hash codes for queries and code snippets and use Hamming distance for rapid recall of code candidates. However, this approach's reliance on linear scanning of the entire code base limits its scalability. To further improve the efficiency of large-scale code retrieval, we propose a novel approach SECRET (Scalable and Efficient Code Retrieval via SegmEnTed deep hashing). SECRET converts long hash codes calculated by existing deep hashing approaches into several short hash code segments through an iterative training strategy. After training, SECRET recalls code candidates by looking up the hash tables for each segment, the time complexity of recall can thus be greatly reduced. Extensive experimental results demonstrate that SECRET can drastically reduce the retrieval time by at least 95% while achieving comparable or even higher performance of existing deep hashing approaches. Besides, SECRET also exhibits superior performance and efficiency compared to the classical hash table-based approach known as LSH under the same number of hash tables.
翻译:代码检索技术根据用户的自然语言描述检索代码片段,被开发者广泛使用并在实际软件开发中发挥着关键作用。深度学习的兴起使检索范式从基于词汇匹配转向利用深度学习模型将源代码和查询编码为向量表示,从而通过向量相似度实现代码检索。尽管这些模型效果显著,但管理大规模代码库仍面临重大挑战。先前研究提出了基于深度哈希的方法,该方法为查询和代码片段生成哈希码,并利用汉明距离快速召回候选代码。然而,该方法依赖对整个代码库的线性扫描,限制了其可扩展性。为进一步提升大规模代码检索的效率,我们提出了一种新方法SECRET(基于分段深度哈希的可扩展高效代码检索)。SECRET通过迭代训练策略,将现有深度哈希方法计算得到的长哈希码转换为多个短哈希码段。训练完成后,SECRET通过为每个哈希码段查找哈希表来召回候选代码,从而大幅降低召回过程的时间复杂度。大量实验结果表明,SECRET在保持与现有深度哈希方法相当甚至更优性能的同时,能够将检索时间至少降低95%。此外,在相同哈希表数量的条件下,SECRET相较于经典的基于哈希表的方法(如LSH)也展现出更优越的性能与效率。