An Early Classification Approach Of Time Series Based On Deep Reinforcement Learning

The problem of early classification of time series(ECTS)is one of the popular research directions in the field of data analysis.By analyzing partially observable time series information,the label of the series can be classified as early as possible.The object of ECTS is to realize the early classification of time series while ensuring the classification accuracy.ECTS is widely used and has many applications in reality.For example,in public transportation,based on historical traffic flow information,early warning can effectively adapt to emergency event with large passenger flow.In the medical field,early classification based on patients' historical observation data can help doctors take treatment measures as early as possible to reduce the threat to patients' lives.Traditional supervised learning-based time series classification models,such as deep learning,can efficiently learn the characteristics of time series so as to classify time series with high accuracy.However,due to the partially observable characteristics of time series,supervised learning cannot decide when to make classification decision,and therefore cannot realize the ECTS.In this thesis,by combining accurate classification and intelligent decision making a novel time series early classification method is proposed.The proposed method takes fully advantage of the intelligent decision-making of reinforcement learning and classification efficiency of deep learning model.In this thesis,a deep learning network was used as a time series feature extractor to fit the state-action value of Q-learning mechanism to form a decision-making procedure made up by reinforcement and deep learning.The reinforcement learning module sent the observation of the environment to the deep learning module,and then Q values of each action were fed back to the reinforcement module to support the decision-making of the agent which made up the early classification model of multivariate time series.By combining sequence category ratio and action ratio,a new experience playback strategy method,is proposed.Moreover,to solve the problem that the model could not converge due to the imbalance of sample category,a reward function concerning with time point and an experience storage method were proposedFinally,the performance of the proposed algorithm is verified on three typical datasets with respect to classification accuracy and earliness performance and realize a simple ECTS system.Experimental results show that by optimizing the uneven experience playback and reward function,1)reinforcement learning model successfully converges,2)and by combining with Q-learning,the new model could make decisions intelligently compared with deep learning ECTS,3)compared with full-length time series classification,time efficiency has increased substantially.Meanwhile,compared with three major early classification models,the proposed method also has a certain improvement on earliness,which can provide a reference for the application of the traditional classification models requiring intelligent decision making.