Visual Image Enhancement Method For UAV Casualty Search And Rescue

The platinum ten minutes and the golden hour are an important principle of casualty search and rescue(SAR).However,the complicated post-disaster and battlefield scene environment restricts rapid deployment of SAR missions,and affects first aid and rehabilitation of casualties.Unmanned Aerial Vehicle(UAV)can approach SAR site quickly and obtain the scene images through visual sensors.Combined with object detection algorithms,the UAV can realize a quick casualty search,and improve the efficiency of casualty SAR mission.However,the visual sensors of UAV are susceptible to interference from external environment,different types of defects will appear in the images,and the performance of detection algorithm tends to decline a lot when running on such defect images.Therefore,three image enhancement models based on deep learning are proposed and applied to the casualty SAR mission from the perspective of UAV.The image enhancement models strengthen the casualty information expression in the defect images and thus improve the object detection algorithm performance on the defect images.The main innovative achievements of this paper are summarized as follows:1.A general progressive image inpainting framework is designed for the incomplete casualty images,which can disassemble complex image inpainting task into several easier sub-tasks.In the framework,a basic image inpainting elemental network is first built,and then each elemental network is connected gradually to form a complete image inpainting network.In order to deepen the network while maintaining stability and improving the image inpainting quality,the framework adopts three construction strategies: multiple access of the to-be-inpainted image to reduce the risk of gradients diffusion,parameter sharing of each elemental network to reduce the model’s weight,and combined loss functions to restrict the inpainted image quality from multiple aspects.A progressive image inpainting network(LPIN)is proposed in this framework,it realizes advanced image inpainting while keeping a concise architecture.Experimental results on public UAV remote sensing images show that the overall accuracy of the scene classification methods combined with LPIN on defect images can reach up to 99% of that on images without defects.The results of self-built Casualty Object Detection Validation Images(CODVI)show that the average precision of the object detection algorithm combined with LPIN on defect images can reach up to 97% of that on images without defects.2.In view of the insufficient semantic inpainting ability of LPIN on images with irregular hole defects,a basic image inpainting elemental network called attention-based semantic inpainting unit(ASIU)is designed.A progressive inpainting generative adversarial network(PIGAN)based on ASIU is constructed within the image inpainting framework proposed in this article.In ASIU,we design a channel attention upsample(CAU)layer to strengthen the model’s attention on reasonable features,and we adopt partial convolution layer to suppress the expression of features in the defect areas.In addition,the generative adversarial architecture ensures a realistic inpainting output of PIGAN.Compared to LPIN and other popular image inpainting models,PIGAN achieves better image inpainting results.Experimental results on self-built CODVI dataset show that the average precision of the object detection algorithm combined with PIGAN on defect images can reach up to 94.5% of that on the images with irregular hole defect,which is higher than 89.1% of LPIN.3.Aiming at the SAR dark casualty images,an unsupervised lightness transfer generative adversarial network(LTGAN)which does not depend on paired dark/bright images and can effectively improve the lightness of dark image is proposed.LTGAN uses the HSL images as additional input to reduce the model’s dependence on RGB images and can prevent color distortion in the enhanced images.A lightness perceptive module(LPM)is designed based on Retinex theory and HLS format image,which forces the model to concentrate on the processing of lightness information.LTGAN achieves good lightness enhancement effect on public dark images,and improves the average precision of casualty object detection algorithm by 15.9% on real dark casualty images.4.To meet the practical needs in real SAR situations,a large-scale casualty search and rescue dataset(CSARD)is set up by simulating casualty SAR from the UAV perspective.On CSARD dataset,we test the detection performance of three combined detection algorithms,which are a combination of our image enhancement models and three general object detection algorithms,namely Faster R-CNN,YOLO v3 and YOLO v4.The results show that the three combined algorithms improve the average precision on defect images by up to 50%.Experiment results on defect videos show that the three combined algorithms reduce the casualty mis-detection rate by up to 31%.The robustness experiment results of combined algorithms on the videos collected at different flight altitudes and speeds show that our image enhancement models still work.To sum up,the defect image of UAV SAR affects the performance of casualty object detection and three image enhancement models are proposed to solve this problem.Firstly,an image inpainting framework for defect images is designed and two image inpainting models,LPIN and PIGAN are constructed in this framework.Secondly,a dark image lightness improvement model LTGAN is constructed for the dark images.Thirdly,a casualty image dataset CSARD from UAV perspective is set up,and evaluation experiments of each image enhancement model are carried out on CSARD.Results show that the three proposed models can significantly improve the casualty detection performance of different object detection algorithms on both defect images and videos,and reduce the casualty mis-detection rate,which lead to the improvement of the robustness of the vision-based casualty SAR mission.