Research On Chinese Sequence Labeling Algorithm Based On BER

With the continuous development of Bidirectional Encoder Representations from Transformers(BERT),Chinese sequence labeling models based on BERT have attracted more and more attention from the academic and industrial communities.Many models have adopted BERT to enhance their labeling performance.This paper mainly focuses on how to use BERT to improve the performance of traditional Chinese sequence labeling models,including the following three subproblems.The first sub-problem is the fusion of characters and vocabulary learning.To address this issue,a BERT model based on cosine similarity adapter is proposed(Cosine-Similarity Adapter BERT,CSBERT).Through cosine similarity,different weights are assigned to relevant vocabulary for each character.This type of BERT can fuse or eliminate feature information of different vocabulary based on the value of cosine similarity.Experiments have shown that CSBERT outperforms other baseline models and this fusion method of character-word feature information has good reference value for other Chinese language processing tasks.The second sub-problem is that some entity boundaries in Chinese sequences may not be clear or ambiguous during representation learning.To address this issue,a boundary-aware vocabulary-enhanced BERT compression(C-BABERT)model is proposed.By introducing a boundary-aware mechanism,the training process of vocabulary-enhanced BERT is optimized with boundary-aware information.The experiment shows that compared to other baseline models,the C-BABERT has a certain performance improvement while saving some training time,especially showing better performance when facing scenarios with unclear entity boundaries.The final sub-problem is the complex relationships between characters or words in Chinese sequences.To address this issue,a vocabulary-enhanced BERT model based on GCN is proposed.By learning the local and global relationships of nodes on the graph through GCN,the complex relationships between characters or words in Chinese sequences can be explicitly captured.Experiments have shown that the LEBERT4 GCN model outperforms other baseline models.Especially when dealing with more complex Chinese sequence data,it is more robust.