Automatic Segmenting And Inpainting Network Based On Two-pathway Network And Attention

Objective:Tissue sectionings play a very important role in the research of histology and pathology.The morphological structure of cells and tissues can be observed and the morphological changes of cell tissues can be judged with sectionings.Clinically,sectionings also reflect the multiple information of tissue cells,making pathological diagnosis more scientific.However,the production of tissue sectionings is difficult and involves many steps.It is inevitable that the sectionings will be damaged during the production process,which makes the slices unusable and affects the research of subsequent experiments.The purpose of this study is to detect and segment the broken area automatically,and leverage neighboring images in sectioning images to restore the broken areas.Methods:In this study,three serial sectioning images are used,which are mice dataset in the 7 days after birth(N7),803 images,mice dataset in embryo 17 days(E17),701 images,and mice dataset in 5 days after birth(N5),413 pictures.This experiment employs a twostage network which is a two-pathway automatic inpainting network based on serial perceive transformer.The inpainting network embedded with image segmentation task could detect the broken areas in the first stage and guide the network to restore the broken area more precisely.Importing Transformer into the image repair field,and leverage selfattention mechanism to extract global feature,gated convolution to learn to extract features from unbroken area and focus on additional texture features from neighboring images to ensure the inpainting result correct.The experiment was trained and tested on the N7 dataset,the generalization ability of the model was tested on the E17 and N5 datasets.Results:Our N7 dataset is divided into training set,validation set and test set with 6:2:2.For broken area segmentation,our model gets great performance with dice 0.9927,accuracy 0.9995,sensitivity 0.9996,specificity 0.9994 and Io U 0.9990.For broken area inpainting,the model could restore tissue structures and details and recognize the tubules which are similar to the broekn areas.As the broken area becomes larger,the inpainting results keeps stable and achieves the metrics of FSIM 0.9478,MS-SSIM 0.9592,PSNR29.7903,VIF 0.8543 and FID 25.2252.Conclusion:We propose a two-pathway automatic inpainting network based on serial perceive transformer which could detect and segment the shape of broken area and restore the broken image automatically.When the broken area is very large,our framework still restores clear and correct details with extracting features from both broken images and neighboring images by serial perceive transformer.The auto inpainting network does not need to take the mask of the broken area as input,which could effectively reduce the workload of annotating,improve the efficiency and accuracy of the restoration,and help the research of basic nephrology.