Semi-Markov CRF has been proposed as an alternative to the traditional Linear Chain CRF for text segmentation tasks such as Named Entity Recognition (NER). Unlike CRF, which treats text segmentation as token-level prediction, Semi-CRF considers segments as the basic unit, making it more expressive. However, Semi-CRF suffers from two major drawbacks: (1) quadratic complexity over sequence length, as it operates on every span of the input sequence, and (2) inferior performance compared to CRF for sequence labeling tasks like NER. In this paper, we introduce Filtered Semi-Markov CRF, a variant of Semi-CRF that addresses these issues by incorporating a filtering step to eliminate irrelevant segments, reducing complexity and search space. Our approach is evaluated on several NER benchmarks, where it outperforms both CRF and Semi-CRF while being significantly faster. The implementation of our method is available on \href{https://github.com/urchade/Filtered-Semi-Markov-CRF}{Github}.

翻译：半马尔可夫条件随机场（Semi-Markov CRF）被提出作为传统线性链条件随机场（Linear Chain CRF）在命名实体识别（NER）等文本分割任务中的替代方案。与将文本分割视为词元级别预测的CRF不同，Semi-CRF将片段视为基本单元，从而更具表达力。然而，Semi-CRF存在两个主要缺陷：（1）序列长度上的二次复杂度，因为它需要处理输入序列的每个跨度；（2）在NER等序列标注任务中性能劣于CRF。本文提出基于过滤的半马尔可夫条件随机场（Filtered Semi-Markov CRF），一种Semi-CRF的变体，通过引入过滤步骤消除无关片段来降低复杂度和搜索空间。我们在多个NER基准数据集上评估该方法，结果表明其不仅在性能上优于CRF和Semi-CRF，同时运行速度显著更快。本方法的实现代码已在GitHub上开源（https://github.com/urchade/Filtered-Semi-Markov-CRF）。