Study Of Aurora Image And Sequence Classification Based On Deep Learning

When the solar wind disturbs the earth's magnetosphere,energetic charged particles from the solar wind and the earth's magnetosphere collide with atoms and molecules in the upper atmosphere,releasing various kinds of visible lights.These visible lights form the gorgeous aurora.Different aurora patterns correspond to different space physics phenomenon.So the study of aurora image and sequence classification is significant.Deep learning technology has developed rapidly in recent years and has been successfully applied to many research fields such as computer vision and pattern recognition.Therefore,in this thesis,the aurora image and sequence classification method are proposed based on deep learning.A multi-size kernels convolutional neural network with eye movement guided initialization is proposed for aurora images classification.Firstly,patches are extracted from the aurora images according to the eye movement information.Then,the features are learned from the extracted patches with the auto-encoder.Furthermore,the convolution kernels of the first convolutional layer are initialized with the learned features.Lastly,a multi-size kernels convolutional neural network is designed.By changing the size of the kernel on the first convolutional layer,we perform the feature learning step at different scales.Experimental results demonstrate that the proposed method can effectively learn features from aurora images and obtain satisfying results on aurora image classification.Although the experts' eye movement information can assist the analysis of aurora images,the collection process of eye movement information is time-consuming.Therefore,it's necessary to learn the experts' cognitive process while observing the aurora images according to the existing eye movement data and predict the experts' eye fixation on aurora images.In this thesis,a method for the prediction of experts' eye fixation on aurora image with fully convolutional network is proposed.Firstly,in order to solve the issue of insufficient training data,the quantized input is used to pre-train the fully convolutional network.Then three kinds of binary ground truth are produced to train the pre-trained fully convolutional network.And three kinds of prediction are generated.Lastly,the conditional random field model is employed for the fusion of different prediction.The aurora is a dynamic and continuous process.So it is not sufficient to analyze the static aurora image only.The temporal information of aurora need to be explored by analyzing the aurora sequence.In order to classify the aurora sequence automatically,in this thesis,we proposed an aurora sequence classification method based on spatial-temporal network.Firstly,the top-down and bottom-up visual attention mechanisms are used to extract the visual attention features of the aurora images.And the visual attention features are fed into the spatial network to analyze the type of the aurora sequence effectively.Then,in the temporal network,the long-short term memory network are used to extract temporal information for aurora sequence classification.Lastly,the final results are obtained by the fusion of the result from the spatial network and the result from the temporal network.Experimental results show that the proposed method outperforms the existing aurora sequence classification methods.