Research On Instance Segmentation For Automatic Medical Scanning Electron Microscopic Particle Image Analysis

In high-throughput drug research,phenotypic analysis of nanoparticles,such as size,shape and surface texture,plays an important role in drug carrier system statistics and has numerous applications.For example,to reduce the burden of nanoparticles on the human digestive system,it is important to select and count particles with diameters below a certain value for the production of drug screening.In practice,the phenotypic analysis of nanoparticles is primarily carried out with the use of medical electron microscope imaging techniques,which scan nanoparticles for imaging.Thereafter,each nanoparticle in the acquired image is segmented.However,compared with deep learning algorithms,traditional image processing methods are time-consuming and labor-intensive,and offer no assistance for automated nanoparticle phenotyping.Therefore,deep learning methods are essential to accelerate the phenotypic analysis of nanoparticles in automated medical electron microscopy nanoparticle images.In computer vision,the instance segmentation task aims to automatically segment each instance of all identical or different objects in an image,which fits perfectly with the pre-phenotyping needs of nanoparticles.In recent years,with the rapid development of deep learning,instance segmentation achieved good results in the processing of natural images as well as some limited application in the analysis of medical images.However,due to the differences between medical electron microscopy particle images and traditional natural images,making them more difficult to obtain for privacy and security reasons.If only a small amount of data is used for network training,the data-based instance segmentation algorithm will struggle to produce the desired results.Additionally,due to the principles of scanning electron microscopy imaging,the images themselves pose many challenges,such as dense and tiny particle situations,varying degrees of blur distortion,overlap,and uneven distribution.As a result,most of the existing instance segmentation methods cannot be directly applied to nanoparticle-related tasks.To address these problems,this thesis develops a one-stage medical electron microscopy particle image instance segmentation algorithm based on local-global aware,taking the characteristics of the electron microscopy particle data itself and the cost as the starting point.The main work and contribution points of this thesis contain the following points.(1)To address the problem that medical electron microscopy particle images are difficult to acquire and lead to small amount of data,this thesis used a combination of weakly annotated data generation and traditional data augmentation.The method first generates a large number of annotated images by domain randomization,and then selects the features which are most similar to the original image features by similarity selection method.Subsequently,to generate final synthetic image by clustering.At the same time,in order to improve the generalization of the subsequent segmentation network,we use traditional data augmentation to increase the amount of real data,and finally,the final synthetic data is combined with the real data to form the required instance segmentation data.(2)In order to address the various problems of nanoparticles in medical electron microscopy nanoparticle images due to the placement and other factors,this thesis presented an one-stage medical electron microscopy particle images instance segmentation based on local-global aware.First,for the case of tiny nanoparticles in electron microscopy images,the algorithm starts from keeping the feature information of tiny particles as much as possible,and extracts feature information directly from the original image to enhance the detail information of tiny particles;meanwhile,this thesis proposes an asymmetric pyramidal network to retain the information of both large and tiny nanoparticles;moreover,in order to make the network focus more on tiny particles,we propose a focal head module,which firstly divides nanoparticles of different sizes into tiny nanoparticles and normal nanoparticles,and then uses a loss function to make it pays more attention on the learning of tiny nanoparticles during segmentation.Secondly,the problem of difficulty in extracting features caused by blurring distortion,low contrast distortion,overlapping,and other issues in electron microscope images,this thesis proposes a local-global context module,the module first divide the electron microscopy particle images into multiple patches with different sizes,and then learn local and global semantic relationships within and between different patches through the local and global context modules.Finally,by reasoning these relationships,a better feature representation is obtained.Finally,to solve the problem of unbalanced data due to distribution of nanoparticles in electron microscopy images,a new loss function is used in our research.In order to evaluate the effectiveness of the proposed algorithm,this thesis validates the proposed algorithm on several existing datasets and obtains several experimental results through several comparative and ablation experiments.These experimental results show that the proposed instance segmentation method in this thesis has a good effect on the problems that existing in medical electron microscopy particle images.