Sequential labeling tasks necessitate the computation of sentence representations for each word within a given sentence. With the advent of advanced pretrained language models; one common approach involves incorporating a BiLSTM layer to bolster the sequence structure information at the output level. Nevertheless, it has been empirically demonstrated (P.-H. Li et al., 2020) that the potential of BiLSTM for generating sentence representations for sequence labeling tasks is constrained, primarily due to the amalgamation of fragments form past and future sentence representations to form a complete sentence representation. In this study, we discovered that strategically integrating the whole sentence representation, which existing in the first cell and last cell of BiLSTM, into sentence representation of ecah cell, could markedly enhance the F1 score and accuracy. Using BERT embedded within BiLSTM as illustration, we conducted exhaustive experiments on nine datasets for sequence labeling tasks, encompassing named entity recognition (NER), part of speech (POS) tagging and End-to-End Aspect-Based sentiment analysis (E2E-ABSA). We noted significant improvements in F1 scores and accuracy across all examined datasets .

翻译：序列标注任务需要对给定句子中的每个词计算句子表示。随着先进预训练语言模型的出现，一种常见的方法是在输出层引入BiLSTM层以增强序列结构信息。然而，经验研究表明（P.-H. Li等人，2020年），BiLSTM在生成序列标注任务的句子表示方面的潜力受到限制，这主要是由于将过去和未来的句子表示的片段合并为完整的句子表示所致。在本研究中，我们发现将存在于BiLSTM首个单元与最后一个单元的整体句子表示，策略性地融入每个单元的句子表示中，可以显著提升F1分数与准确率。以嵌入BiLSTM中的BERT为例，我们在九个序列标注任务数据集上进行了详尽的实验，涵盖命名实体识别（NER）、词性标注（POS）以及端到端基于方面的情感分析（E2E-ABSA）。我们注意到在所有考察的数据集上，F1分数和准确率均有显著提升。