Deep Learning Based Method For Auroral Local Structure Recognition And Localization

Aurora is a spectacular phenomenon that appears around the high latitude area of the earth.The light is emitted by atmospheric atoms and molecules that have been excited by collisions with electrons and protons that precipitate into the atmosphere from outer space.As an optically thin projection screen reflecting the solar activities and changes in the Earth's magnetosphere,the aurora is an important way to monitor and investigate the physical processes in near-Earth space for geosciences.The morphological types of aurora have turned out to be correlated with specific magnetospheric regimes and dynamic activity and influenced by the solar wind parameters.The study of auroral morphology and its evolution process not only helps to reveal the solar wind-magnetosphere-ionospheric coupling process and its internal mechanism,but also provides an important physical basis for space weather forecasting.It is of great significance to improve the accuracy of polar area communication and navigation systems.Of various facilities for aurora observation,the ground-based optical all-sky imaging is an important way to obtain and record the auroral morphology,since it can consistently capture the two-dimensional morphological information with high spatial and temporal resolution.To effectively study the auroral morphology on a big auroral image database,researchers have developed many automatic analysis methods based on machine learning and pattern recognition,such as auroral image retrieval,auroral image classification,and auroral image segmentation.However,due to the complex auroral morphology,the recognition accuracy of the existing methods is still low and depends on a large number of training samples.More importantly,the recognition of the local structure and the analysis of its physical parameters are very important for studying the physical processes of the aurora,while existing methods cannot achieve automatic recognition and numerical analysis of auroral local structures.To this end,we study weakly,semi,and unsupervised deep learning methods for the local structure localization and auroral image classification methods.In this dissertation,the recognition accuracy of auroral morphology is significantly improved,the demands for the number of labeled samples are reduced in different levels,the automatic recognition and physical parameter computation of auroral local structure is achieved,and the feasibility and effectiveness of the proposed methods are proved on the all-sky auroral images.The main contributions of this dissertation are summarized as follows:(1)A weakly-supervised semantic segmentation(WSSS)method is proposed to achieve joint key local structure localization and classification of the auroral image.To improve the classification accuracy of auroral images,we studied the recognition process of aurora experts and found that the aurora experts tend to first localize the local structures in the different size of field-of-view and then classify the aurora image based on these local structures.Based on this observation,we define the local structures that determine a certain aurora type as the key local structure(KLS),and propose a semantic segmentation method to achieve joint KLS localization and classification of auroral images.Since it is of great difficulty to obtain large amounts of pixel-level annotated aurora images,we propose a weakly-supervised semantic method that requires image-level labels only.According to the unique characteristics of the aurora,we develop a patch scale model(PSM)and a region scale model(RSM)for analyzing the low-level details features and high-level overall arrangement features respectively.The PSM and RSM together achieve joint KLS localization and auroral image classification.The proposed method improves the classification accuracy by 5.5% in comparison with the traditional deep convolutional neural networks(DCNN)and improves the accuracy of KLS localization by 9.1% in comparison with the state-of-the-art WSSS methods.Based on the above study,we further propose a semi-supervised semantic segmentation method for auroral image classification.By using an extra small number of pixel-level annotations,the classification of auroral images is improved by 8.5% in comparison with the traditional DCNN.(2)A semi-supervised semantic and instance segmentation method is proposed for reducing the pixel-level annotations.As the existing semantic and instance segmentation methods require a large number of pixel-level annotations and thus they are hard to apply in many situations,we propose a semi-supervised model that is trained with a large number of box-level annotations and a small number of pixel-level annotations.We decompose the semantic and instance segmentation into three components: object detection,attention,and segmentation.First,the object detection model is trained with box-level annotations only to achieve object recognition and coarse localization.Second,the attention module can transfer the semantic segmentation to the binary segmentation by generating class-specific features.Finally,a binary segmentation model shared with all classes is trained with pixel-level annotations only.By simplifying the segmentation task,the proposed method can reduce the requirements of pixel-level annotations.The experimental results on VOC 2012 show the effectiveness of the proposed method.(3)A fully automatic auroral arc width computation method based on instance segmentation is proposed.As the exiting the automatic method for all-sky auroral images cannot achieve the auroral arc width computation in a fully automatic manner,this dissertation is the first study to achieve fully automatic auroral arc width computation based on instance segmentation.To improve the instance segmentation accuracy of the auroral arc,we design a two-stage inference process and a random rotation strategy.The proposed method significantly improves the instance segmentation accuracy by 20+% in comparison with the state-of-the-art Mask R-CNN method.Based on the instance segmentation results,an auroral arc width computation method is designed.We applied the proposed method on 18417 auroral arc images and obtained similar results to the human interactive method,which demonstrates the effectiveness of the proposed method.To simultaneously recognize the auroral arc and compute the arc width,we further integrate an arc recognition module and design a random training strategy.The experiments on the auroral image datasets that contain various aurora types demonstrate the effectiveness of the proposed method.(4)A self-supervised attention network is proposed for unsupervised clustering.As the existing deep clustering methods based on autoencoder are hard to extract discriminative features,we propose a semantic clustering method based on self-supervised learning,which directly outputs cluster labels without extracting intermediate features.To tackle the problems that avoiding trivial solutions,capturing object semantics,and processing large images,we propose an entropy loss,an attention mechanism,and a two-step learning algorithm,respectively.The experimental results on 5 natural datasets show that the clustering results on three metrics are improved by 8%,7% and 10% on STL10 compared with the state-of-the-art methods,the clustering accuracy is improved by 5% on Image Net-10,and the proposed method achieves competitive results on other datasets.By applying the proposed method to the auroral image,the performance of unsupervised auroral image clustering is significantly improved.