Research On In-situ Electron Microscope Image Recognition And Tracking Algorithm Based On Deep Learning

In-situ electron microscope can be used to observe the dynamic chemical reaction process of nanocatalyst under reaction atmosphere.However,after obtaining massive video data onto electron microscope,it is very important to mine the information(Size,location,etc.)of nanoparticles over time for the video data onto the whole reaction process,and it is also a necessary topic to avoid the unnecessary discard of high-value data.At present,the structure information statistics and analysis of nanoparticles are mostly carried out by manual identification of electron microscope images.The manual methods are time-consuming and has few identification samples.It is difficult to reflect the actual structure information on a large scale objectively and accurately and to track the dynamic changes of nanoparticles.Deep learning technology has excellent performance in many machine vision fields with its powerful automatic extraction and nonlinear fitting ability,such as medicine,biology,and automatic driving.This dissertation focuses on the diversity and complexity of the microstructure characteristics(Shape,size,element distribution,etc.)of nanocatalysts,and applies deep learning to the field of in-situ electron microscope image analysis of nanocatalysts.The instance segmentation and tracking algorithm based on deep learning was used to complete the identification,segmentation and tracking of nanoparticles in the video data of in-situ electron microscope.According to the sequential microscopic data of in-situ electron microscope,the dynamic structural change information of catalysts related to the extraction reaction was identified,which provided the basis for researchers to establish the correlation between catalytic performance and structure.Firstly,for the in-situ electron microscope image dataset with unbalanced positive and negative samples,the data augmentation technology is used to expand the training dataset.Then,the performance of two instance segmentation algorithms based on two-stage MASK RCNN and one-stage YOLACT is compared.On this basis,a MASK RCNN network based on depth separable convolution is proposed,which is more suitable for the hardware conditions of insitu electron microscope.Under the premise of ensuring the segmentation accuracy,the network is lighter and the prediction time is shorter.Then,based on the accurate segmentation of nanoparticles,the target tracking algorithm is used based on the open source trackpy software package to obtain the trajectory and structural change information of each particle.According to the identification and tracking results,the key structural information such as shape,size and position of the catalyst were counted,which was correlated with the actual catalytic performance measured by in-situ electron microscope and the structure-activity relationship was established.Finally,a graphical user interface for nanoparticle recognition and tracking is developed based on Pyqt to facilitate the use and statistics of useful information by researchers in the field of nanomaterials.Compared with the traditional manual labeling method,the proposed method has been significantly improved in efficiency,accuracy and convenience,which lays the foundation for the application of deep learning technology in the field of catalyst electron microscope image analysis.In addition,the electron microscopic characterization technique was extended from microscopic observation to quantitative statistics of macroscopic results,and then the activity contribution of different microstructures in the catalyst was analyzed.The structure-activity relationship of the actual catalytic system was revealed from the molecular and atomic structure of the material.The research work of this dissertation has certain guiding significance for the rational design of catalysts,and provides a useful attempt and exploration for the online application of deep learning in in-situ electron microscope image analysis.