Lempel-Ziv (LZ77) factorization is a fundamental problem in string processing: Greedily partition a given string $T$ from left to right into blocks (called phrases) so that each phrase is either the leftmost occurrence of a letter or the longest prefix of the unprocessed suffix that has another occurrence earlier in $T$. Due to numerous applications, LZ77 factorization is one of the most studied problems on strings. In the 47 years since its inception, several algorithms were developed for different models of computation, including parallel, GPU, external-memory, and quantum. Remarkably, however, the complexity of the most basic variant is still not settled: All existing algorithms in the RAM model run in $\Omega(n)$ time, which is a $\Theta(\log n)$ factor away from the lower bound of $\Omega(n/\log n)$ (following from the necessity to read the input, which takes $\Theta(n/\log n)$ space for $T\in\{0,1\}^{n}$). We present the first $o(n)$-time algorithm for LZ77 factorization, breaking the linear-time barrier present for nearly 50 years. For $T\in\{0,1\}^{n}$, our algorithm runs in $\mathcal{O}(n/\sqrt{\log n})=o(n)$ time and uses the optimal $\mathcal{O}(n/\log n)$ working space. Our algorithm generalizes to $\Sigma=[0..\sigma)$, where $\sigma=n^{\mathcal{O}(1)}$. The runtime and working space then become $\mathcal{O}((n\log\sigma)/\sqrt{\log n})$ and $\mathcal{O}(n/\log_{\sigma} n)$. To obtain our algorithm, we prove a more general result: For any constant $\epsilon\in(0,1)$ and $T\in[0..\sigma)^{n}$, in $\mathcal{O}((n\log\sigma)/\sqrt{\log n})$ time and using $\mathcal{O}(n/\log_{\sigma}n)$ space, we can construct an $\mathcal{O}(n/\log_{\sigma}n)$-size index that, given any $P=T[j..j+\ell)$ (represented as $(j,\ell)$), computes the leftmost occurrence of $P$ in $T$ in $\mathcal{O}(\log^{\epsilon}n)$ time. In other words, we solve the indexing/online variant of the LZ77 problem.
翻译:Lempel-Ziv (LZ77) 因子分解是字符串处理中的一个基本问题:将给定字符串 $T$ 从左至右贪婪地分割为块(称为短语),使得每个短语要么是某个字母的首次出现,要么是未处理后缀的最长前缀,且该前缀在 $T$ 中更早的位置出现过。由于其广泛的应用,LZ77 因子分解是字符串领域研究最多的问题之一。自其提出 47 年以来,针对不同的计算模型(包括并行、GPU、外存和量子计算)已开发出多种算法。然而,值得注意的是,最基本变体的计算复杂度至今仍未确定:RAM 模型中所有现有算法的运行时间均为 $\Omega(n)$,这比 $\Omega(n/\log n)$ 的下界多了一个 $\Theta(\log n)$ 因子(该下界源于读取输入的必要性,对于 $T\in\{0,1\}^{n}$,输入占用 $\Theta(n/\log n)$ 空间)。我们提出了首个 $o(n)$ 时间的 LZ77 因子分解算法,打破了近 50 年来存在的线性时间壁垒。对于 $T\in\{0,1\}^{n}$,我们的算法在 $\mathcal{O}(n/\sqrt{\log n})=o(n)$ 时间内运行,并使用最优的 $\mathcal{O}(n/\log n)$ 工作空间。我们的算法可推广到 $\Sigma=[0..\sigma)$,其中 $\sigma=n^{\mathcal{O}(1)}$。此时的运行时间和工作空间分别变为 $\mathcal{O}((n\log\sigma)/\sqrt{\log n})$ 和 $\mathcal{O}(n/\log_{\sigma} n)$。为了得到我们的算法,我们证明了一个更一般的结果:对于任意常数 $\epsilon\in(0,1)$ 和 $T\in[0..\sigma)^{n}$,我们可以在 $\mathcal{O}((n\log\sigma)/\sqrt{\log n})$ 时间内,使用 $\mathcal{O}(n/\log_{\sigma}n)$ 空间,构建一个大小为 $\mathcal{O}(n/\log_{\sigma}n)$ 的索引,该索引给定任意 $P=T[j..j+\ell)$(表示为 $(j,\ell)$),即可在 $\mathcal{O}(\log^{\epsilon}n)$ 时间内计算出 $P$ 在 $T$ 中的最左出现位置。换句话说,我们解决了 LZ77 问题的索引/在线变体。