Application Of Transformer-based Object Detection And Semantic Segmentation Models For Remote Sensing Images Of Coal Mines

Coal resources are an important part of China’s energy structure.There are approximately 4,000 coal mines still in operation.Coal mines produce many air pollutants,such as methane,during production and mining.Obtaining the distribution of coal resources in China and studying the regional variations of coal mining are not only useful for the development of energy planning in China,but also important for monitoring the location and estimating the concentration of pollutants.However,the spatial distribution of coal mines is still obtained by manual survey or by referring to old databases,which not only consumes huge workload,but also cannot be updated quickly.At present,deep learning models have been widely used in various studies of remote sensing images,and the object detection and semantic segmentation in deep learning can accurately find or extract the required feature categories in remote sensing images,but there is a lack of extraction methods for coal mines,especially Underground coal mines,on remote sensing images.In this study,we firstly construct two datasets of coal mine and opencast coal mines based on Google Earth images using the existing list of coal mine coordinates.For the underground coal mines,which are distinct from other features but have unclear boundaries,a target detection model is trained on this dataset.For the opencast coal mine,where the texture and color are distinctly different from other features and the boundary is clear,a semantic segmentation model is used to train on this dataset.The trained target detection model can detect the coal mine on the remote sensing image,and the semantic segmentation model can successfully extract the opencast coal mining area on the remote sensing image.And the detection of coal mine on the high-resolution remote sensing image(GF-1/2)was achieved by means of GDAL chunk reading image and CUDA accelerated code combined with the target detection model.The mask generated by using the semantic segmentation model combined with the image scale can count the proportion of the change area of the mining area of the open pit coal mine at the same location under different time.The convolutional neural network(CNN)among the deep learning models is good at extracting local features as the most commonly used feature extraction module,but lacks the ability to capture the global features of remote sensing images.The Transformer model,on the other hand,can obtain global contextual information on the image through a self-attentive mechanism to obtain more advanced features to improve the model learning ability.Therefore,in this thesis we construct the improved target detection model Swin transformer-YOLOv5 and the improved semantic segmentation models Swin-UNet and Trans UNet based on Transformer,and train them on the wellworking coal mine dataset and the open-pit coal mine dataset constructed in this study,respectively,and then compare them with each other and the mainstream deep learning models The results are as follows:(1)The m AP of Swin transformer-YOLOv5 is 3.9%higher than that of the base version YOLOv5 in the target detection task of the wellworking coal mine,and the m AP value is the highest among the mainstream target detection models.(2)The MIo U value of Trans Unet with a mixture of CNN and Transformer in the open-air coal mine semantic segmentation task is 5.73% higher than that of the UNet model,while the MIo U value of Swin-Unet with only Transformer decreases by 10.91%,which indicates that the deep learning model can be improved by combining with Transformer in combination with Transformer to improve its learning ability and model performance.This thesis has 39 figures,16 tables,and 88 references.