Sequential labeling tasks necessitate the computation of sentence representations for each word within a given sentence. With the advent of advanced pretrained language models; one common approach involves incorporating a BiLSTM layer to bolster the sequence structure information at the output level. Nevertheless, it has been empirically demonstrated (P.-H. Li et al., 2020) that the potential of BiLSTM for generating sentence representations for sequence labeling tasks is constrained, primarily due to the amalgamation of fragments form past and future sentence representations to form a complete sentence representation. In this study, we discovered that strategically integrating the whole sentence representation, which existing in the first cell and last cell of BiLSTM, into sentence representation of ecah cell, could markedly enhance the F1 score and accuracy. Using BERT embedded within BiLSTM as illustration, we conducted exhaustive experiments on nine datasets for sequence labeling tasks, encompassing named entity recognition (NER), part of speech (POS) tagging and End-to-End Aspect-Based sentiment analysis (E2E-ABSA). We noted significant improvements in F1 scores and accuracy across all examined datasets .

翻译：序列标注任务需要针对给定句子中的每个单词计算句子表示。随着先进预训练语言模型的出现，一种常见方法是在输出层引入BiLSTM层以增强序列结构信息。然而，实证研究（P.-H. Li等人，2020年）表明，BiLSTM在生成序列标注任务句子表示方面的潜力受到限制，主要原因是其将过去和未来的句子表示片段混合形成完整句子表示。本研究发现，将存在于BiLSTM首个单元和最后一个单元中的完整句子表示，策略性地融入每个单元的句子表示中，能够显著提升F1分数和准确率。以嵌入BiLSTM的BERT为例，我们在涵盖命名实体识别、词性标注和端到端方面级情感分析的九个序列标注任务数据集上进行了全面实验。结果显示，所有检验数据集的F1分数和准确率均取得显著提升。