The Representation And Recognition Of Trajectory Data Based On Path Signature Feature And Deep Learning Methods

The ubiquitous trajectory data,including handwritten characters,handwritten signatures,hand gestures,human actions,traffic vehicle flows,etc.,have a really close relationship with human beings with respect to our daily life,personal safety and property security.Due to the fast development of motion sensors in intelligent devices,an increasing number of trajectory data can be easily captured,which facilitate the research of trajectory data mining and analysis.As a key step of trajectory data mining,trajectory recognition aim to achieve automatic recognition using machine learning techniques and arise the value of the large-scale captured raw trajectory data.In this paper,we focus on handwritten trajectory which is the movement of single object,and human action trajectory which contains multiple moving objects at the same moment.Leveraging the path signature features from rough path theory and the deep learning methods,we explore novel trajectory feature representation and classification methods which are robust,efficient and effective.Currently,trajectory recognition tasks have many challenges.First,the trajectory data have a great deal of diversity,such as large-scale,multi-classes,noised labelling,etc.,posing many chanllenges to the training of deep models.Second,the feature representation of trajectory is often required to be able to capture both local and regional and global information to ensure the completeness of representation.Last but not least,for high dimentional multi-object movement,one should find a way to combine information from different dimention of raw trajectory,so that the spatial structure and the temporal dynamics can be well described for further recognition.To solve these probloms,the main contributions and outcomes of this paper are listed below:(1)To deal with the complexity of trajectory data,we proposed various novel solutions to data enhancement,efficient data training,and data augmentation.For handwritten characters,we designed a domain-knowledge-enhanced layer,which considers several traditional techniques of handwriting recognition domains and extracts rich prior knowledge,to improve the performance of deep models.To cope with the large-scale multi-class handwritten characters,we were inspired by the Leither learning box from psychology and designed a novel deep learning traing strategy,namely DropSample.Instead of uniform sampling in the training process,DropSample strategy uses the temporal output of deep models as a confidence measue to automatically and dynamically adjust the probability of samples being selected.The samples which are difficult to recognize will be frequently reviewed to boost the training process,and those which are mistaken labeling will be discarded to prevent involving harmful effect.Based on these methods,we eventually achieved many state-of-the-art results in handwritten recognition tasks.For writer identification task,given the dataset is small,we proposed a data augmentation method,called DropSegment,which randomly removes a proportion of detected segments from the whole trajectory at each training iteration to create a large amout of pseudo characters.Moreover,the DropSegment method abandons the original structure of trajectory so that our system can be robust against the various text contents.(2)As it is important to extract both local and global features for trajectory representation,we proposed different variants of path signature features.We extensively explained the path signature and its basic properties from the prospective of application,and described the protential ability of path signature as trajectory features.For rotation-free handwriting recognition,we proposed dyadic path signature,which transforms the entire trajectory into hierarchical small pieces before feature extraction,so that the signature over these pieces can involve information of all ranges from local to global.For writer identification,the key information for distinguishing different writers' handwritings conceals in fine-grained details,we therefore extracted high order path signature features to capture the local geometic dependences and finally achieved breakthrough in writer identification.(3)Regard to the high-dimensional multi-object moving trajectory data,we proposed a spatio-temporal path signature feature extraction method.For human skeleton-based action recognition,we grouped the skeletal joints into pairs and triples over which the signature can represent spatical structural information.Then,the evolution of these spatical features along the time can also be regarded as path over which the signature describe the temporal dynamics.Finally,we achieved state-of-the-art recognition results in four typical human action datasets.