Tensor-Based High-Order Long-Short Term Memory Network(LSTM)Models

With the development of information technology,human living environment gradually is becoming hybrid information contained cyber world,physical world and social world,called ”Cyber-Physical-Social System”(CPSS).As a new cross-disciplinary area,data collected from CPSS mapping the historical trace of users,and contains huge value.how to deal CPSS data to provide better service is one of main research directions.Meanwhile,CPSS big data have three main features as following.Firstly,data from CPSS usually hava multi-order and multi-dimension since the process of collecting data contained the feature of three system.Secondly,the process of collecting CPSS data contains redundancy and noisy.Thirdly,data contained temporal data stream and real time data.CPSS big data is represented by tensor because of its characteristic of contained multi-dimensional information.Meanwhile,artificial intelligence(AI)provide new idea to process CPSS big data.Long Short-Term Memory(LSTM),as an efficient artificial intelligence method for temporal data,has been widely application.Since CPSS big data contained time series data,tensor-based LSTM was proposed and was used in some area.First of all,this thesis proposes Tensor-based Long Short-Term Memory Based Quantized Tensor-Train(QTT-LSTM).QTT-LSTM make the component of LSTM to tensor for retain the multi-attribute features of CPSS big data.And QTT-LSTM represents the weight in the form of QTT chain,which effectively reduces the required parameters of LSTM network and the difficulty of training.Next,considering the redundancy of CPSS big data and the improvement of edge computing node,this thesis proposes Cloud-edge-aided Quantized Tensor-Train Distributed Long Short-Term Memory(QTT-DLSTM).QTT-DLSTM contains edge plane and cloud plane.In the edge plane,the work is data preprocessing.It is efficient to feature extraction and compression for data.In the cloud plane,the work contains data fusion from each edge plane and training in specially designed neural network.It can effectively reduce training difficulty and get passable accuracy.Finally,experiments are carried out on two industrial data sets(manufacturing industry data set and Battery Aging ARC data set)and one video data set(UCF50),and are analyzed.It can get best accuracy and better compression ratio on two industrial data sets and reduce one order of magnitude within 3% accuracy error on UCF50.