Pansharpening And Image Generation Of Remote Sensing Disaster Images Based On Deep Learning

The frequent occurrence of disasters has seriously threatened the safety and property of people.Rapid detection and assessment after the occurrence of disaster play a very important role in humanitarian assistance and disaster recovery.In recent years,with the development of deep-learning techniques and the wide applications of remote sensing images,research on pansharpening and disaster detection has made rapid progress.However,it is still a great challenge to train an accurate and robust disaster detection network,due to the class imbalance of existing datasets and the lack of training data.Thus,this paper aims at solving the problems existing in remote sensing data,including the spatial resolution of multispectral images and the limit in disaster images.We propose a mask guided pansharpening network and a disaster image generation method.The main work are as follows:(1)Due to physical limitations of satellite sensors,we can not get remote sensing images with high spatial resolution and high spectral resolution simultaneously.In this paper,we propose a mask guided pansharpening network(MaskPan)with the goal of improving the visual quality and visual discrimination.MaskPan adopts MaskLayer to guide pansharpening,which can make full use of semantic information.Moreover,MaskPan fuse the spectral information and spatial information from multispectral images and panchromatic images in feature domain instead of pixel domain,attention mechanism is adopted to achieve effective fusion.To verify the effectiveness of the proposed method,we compare MaskPan with the classical pansharpening methods.Qualitative and quantitative experiments demonstrate that the proposed methods is superior to the other pansharpening methods from the aspect of the image quality and visual discrimination.(2)Besides,we aims at synthesizing disaster remote sensing images with multiple disaster types and different building damage with GANs,making up for the shortcomings of the existing datasets.However,existing models are inefficient in multi-disaster image translation due to the diversity of disaster and inevitably change building-irrelevant regions caused by directly operating on the whole image.Thus,we propose two models,Disaster Translation GAN can generate disaster images for multiple disaster types using only a single model,which adopt a attribute to represent disaster types and the reconstruction process to further ensure the effect of generator.In addition,Damaged Building Generation GAN is a mask-guided image generation model,which can only alter the attribute-specific region while keeping the attribute-irrelevant region unchanged.Qualitative and quantitative experiments demonstrate the validity of the proposed methods.Further experimental results on the disaster detection model show that the proposed models can be used as an effective data augmentation strategy to improve the accuracy of the detection model.To sum up,this paper explores the pansharpening and image generation of remote sensing disaster images.The proposed method improves the spatial resolution of multispectral images,and generates a variety of remote sensing disaster images,which pro vides support for further exploration of remote sensing disaster detection.