Multiple pattern matching in strings is a fundamental problem in text processing applications such as regular expressions or tokenization. This paper studies efficient implementations of double-array Aho-Corasick automata (DAACs), data structures for quickly performing the multiple pattern matching. The practical performance of DAACs is improved by carefully designing the data structure, and many implementation techniques have been proposed thus far. A problem in DAACs is that their ideas are not aggregated. Since comprehensive descriptions and experimental analyses are unavailable, engineers face difficulties in implementing an efficient DAAC. In this paper, we review implementation techniques for DAACs and provide a comprehensive description of them. We also propose several new techniques for further improvement. We conduct exhaustive experiments through real-world datasets and reveal the best combination of techniques to achieve a higher performance in DAACs. The best combination is different from those used in the most popular libraries of DAACs, which demonstrates that their performance can be further enhanced. On the basis of our experimental analysis, we developed a new Rust library for fast multiple pattern matching using DAACs, named Daachorse, as open-source software at https://github.com/daac-tools/daachorse. Experiments demonstrate that Daachorse outperforms other AC-automaton implementations, indicating its suitability as a fast alternative for multiple pattern matching in many applications.
翻译:字符串中的多模式匹配是文本处理应用(如正则表达式或分词)中的基本问题。本文研究双数组Aho-Corasick自动机(DAACs)的高效实现,这是一种用于快速执行多模式匹配的数据结构。通过精心设计数据结构可提升DAAC的实际性能,目前已有多种实现技术被提出。DAAC面临的一个问题在于其技术思路尚未系统整合。由于缺乏全面的描述和实验分析,工程师在实现高效DAAC时面临困难。本文梳理了DAAC的实现技术并提供了全面描述,同时提出若干新技术以进一步优化性能。我们使用真实数据集进行详尽实验,揭示了实现DAAC高性能的最佳技术组合。该最佳组合与当前最主流DAAC库所采用的技术不同,表明这些库的性能仍有提升空间。基于实验分析,我们开发了名为Daachorse的新型Rust库(开源地址:https://github.com/daac-tools/daachorse),用于利用DAAC实现快速多模式匹配。实验表明,Daachorse的性能优于其他AC自动机实现,证明其适合作为众多应用中多模式匹配的快速替代方案。