Research On Sentence Embedding Method For Multitasks

Natural language processing(NLP)is a technology that applies language implication information in natural language to enable machine the ability of capturing real meanings.The development of NLP plays a key role in machine comprehension and bridges the communication connection between human and machine.Vectorization is a hot topic in NLP recent years,which contains word vectorization and sentence vectorization.Word2vec,glove and other word embedding methods are produced in word vectorization expression.Word embedding largely stimulates the booming of applications from artificial intelligence and is widely utilized in all NLP areas.Non-transferable feature in sentence embedding seriously hinders training and transfer between models.Hence,obtaining the general transferable models is huge focus in NLP.Under this circumstance,our project puts forward the idea of supervised sentence-embedding,which is a general transferable model which should be well trained in specific tasks with sentence semantics and gain high performance with transfer ability towards multiple tasks.Two major aims are listed as follows:(1)We introduce the two-type supervised sentence embedding context association models:one is context information pooling model,by applying recurrent neural network in mapping context,we obtain the denoised abundant vector features as an input for max pooling to sentence embedding;the other one is context information attention model,it is based on attention model along with context information for integration to fix dimension sentence embedding with every timestamp feature information.Validation is based on SNLI dataset.Furthermore,according to word multi-meanings we put forward the word semantic masks model and valid its availability and limitations.(2)Compare performance results between our model and currently state-of-art sentence embedding models in transfer tasks.Sentence embedding is clustered to 3 types:ordered information unsupervised model,un-ordered information unsupervised model and supervised model.Context information association model performed best in transferability through multiple tasks.According to results from comparing study,it proofs the availability to refine and reflect whole sentence information in recurrent neural network with attention model on specific and pooling mechanism is leading among models with strengthened adaptability.