Research And Implementation Of Semantic Cognitive Text Encoder

Artificial intelligence mainly includes three different levels of computational intelligence,perceptual intelligence,and cognitive intelligence.At present,the realization of cognitive intelligence has become a popular direction in academics and industries.Since the results of text encoding can be applied to a variety of natural language processing tasks such as text classification,sentiment analysis,syntax analysis,and automatic question answering,the design of text encoders with semantic cognitive intelligence have become a hot topic currently.This topic first combines the current status of research at home and abroad,and summarizes two major difficulties in implementing a semantic cognitive encoder-polysemic uncertainty and task uncertainty.In recent years,deep learning has become the mainstream of text encoders,but neither the existing convolutional neural network nor the recurrent neural network can handle the problems of polynomial uncertainty and task uncertainty well.Beginning in June 2018,Google’s pre-trained network BERT appeared,and more pretrained models such as XLNet and T5 were proposed in 2019,which can better solve the problem of task uncertainty,but the amount of parameters in these models is much larger than others and they require a large number of pre-trained datasets,which cannot be applied on real-time systems(such as online learning systems).Aiming at overcoming the shortcomings of existing text encoders,this thesis proposes two novel and practical text encoders,which are Multi-prototype based semantic cognitive encoders and pre-trained based semantic cognitive encoders.The former has a small amount of parameters and training cost,and I have proof its effectiveness through experiments that it outperforms the deep learning networks of similar amount of parameters.The latter relies on the BERT model,and further improves its performance on various datasets based on the BERT.With no limit on the amount of parameters,the latter’s coding effect will be better than the former.The structures of both encoders can deal with the two major problems of polysemy uncertainty and task uncertainty through effective encoding methods.Among them,both introduced the deconvolutional neural network,which rarely occurs in Natural Language Processing tasks,as part of the text encoder,which can effectively deal with the problem of polysemy uncertainty.At the same time,the Multi-prototype-based semantic cognitive encoder effectively solves the task uncertainty problem through the end-to-end training network structure.In order to analyze whether the two encoders can specifically deal with the two problems of polysemy uncertainty and task uncertainty,this thesisalso introduces two methods of text deconvolution saliency test and singular value decomposition.By visualizing the encoding results of the two encoders,this thesis futher verifies the semantic cognition ability of the two encoders is verified.This thesis selected three data sets,named the General Language Understanding Evaluation(9 tasks in total),the text classification dataset(3 tasks in total),and the SQuAD data set.As is shown in the experiments,the performance of the two semantic cognitive encoders on each dataset outperforms all the baseline encoders,and the practicality and effectiveness of the semantic cognitive encoders are verified from an application perspective.