Research Of Key Technologies For Position-based Text Matching

Position-based text matching approaches embed the positional information of the context into similarity models,which can better model the sequential and structural fea-tures of a text and have shown great advantages in many natural language processing tasks.The position-based text matching models focus on two issues:(1)the measure-ment of the positional information;(2)the combination of the positional information and the similarity models.Previous work usually adopted kernel functions to model the position-based interac-tion features.They usually pre-defined a kernel for all words in the collection.However,those kernel functions need to be tuned according to experience in different documents and cannot satisfy the requirements in various circumstances.How to automatically gen-erate the most suitable kernels for different words is still not been well studied.Moreover,the recent deep learning algorithms have shown their excellent performance in text rep-resentation.However,it is hard to incorporate the positional information into the deep methods,due to the complexity and un-interpretability of the neural networks.Previous work on position-based neural networks usually directly took the positional features as the inputs of the networks.How to use the positional information during the network structure building are also not been well studied.In this paper,we make a depth investigation on kernel function selection and position-based neural network construction.As for the kernel selection,we make full use of the statistics information of words and the probability theory to implement an intelligent ker-nel selection approach,which can automatically adjust itself to different circumstances.Regarding the position-based neural network,we first classify the positional information into two categories,namely temporal position and spatial position.Then,we apply them into attention gating mechanism,hierarchical structures,and fractional calculus for better text similarity modeling.Specifically,the contributions of this paper are:1.We propose an intelligent kernel selection approach for positional informa-tion measurement.In this paper,the concept of elite kernel function is defined to identify and measure the positional influence distributions of technical terms in d-ifferent circumstances.The proposed elite kernel selection approach is designed by using the probability theory,which requires no extra resource and pre-training step and has strong compatibility and stability in different position-based text matching models.It is notable that,the intelligent selection approach breaks the limitation of unsupervised learning in traditional text matching models.2.We enhance the recurrent neural network with a positional gating mechanism.In this paper,the sequential features of co-occurring words in a text pair is defined as the temporal position information.We use the temporal position information to generate a gating mechanism,which controls information flow between the text pair during text representation.This mechanism can effectively reduce the influence of noise on text similarity calculation.In addition,it can also enhance the effect of word interactions between the text pair during text representation.3.We propose a positional convolutional neural network for text matching by ab-sorbing the spatial position features.We classify the positional information into three categories,namely word level,phrase level,and sentence level,and incorpo-rate them in a similarity function for multi-perspective text matching.Particularly,we design a position-sensitive convolution filter to recognize and extract positional information at different levels from the text.Different from the traditional random-ly generated convolution filters,the proposed filter is generated by a kernel density function,which further provides a new and feasible research field for the convolu-tion neural network.4.We propose a fractional latent topic based neural network for text matching,which incorporates the temporal and spatial position information in a similar-ity function.Particularly,we introduce the fractional calculus into text processing for the latent topic generation in spatiotemporal level,which can better cope with the issues of polysemy and ambiguity.To the best of our knowledge,this is the first attempt to use the fractional calculus for natural language processing,which pro-vides an avenue for the fractional calculus theory application in the text field with the theoretical and practical supports.