Research On Key Technologies Of Cloud Detection And Restoration For Optical Remote Sensing Images

Optical remote sensing image are one of the most important data sources for Earth observation.They have been widely applied in many fields such as land cover mapping,vegetation and water monitoring,mineral and oil detection,precision agriculture,and so on;which has provided irreplaceable support for global environmental monitoring,environmental resource management and ecological management.However,optical remote sensing images are subject to the passive imaging mechanism and is inevitably be affected by cloud and cloud shadow occlusion,forming invalid pixels with low or no usability in the image and resulting in the coverage or distortion of the ground information and finally affecting image interpretation and analysis.Most remote sensing data processing systems require the proportion of invalid pixels to be less than 15%,causing a large number of cloudy images to be completely wasted.Therefore,it is necessary to detect clouds and shadows in the images,and repair the invalid pixels to restore the overall image consistency and improve per-pixel utilization of cloudy images.Existing researches mainly focus on the detection and repair of a single image,ignoring the correlation of the time dimension of satellite images.With the accumulation of historical observation data of existing satellite sensors and the development of micro-nano remote sensing images,apart from the spatial dimension,there will be a large number of remote sensing images that can be analyzed and processed in the time dimension.It is of great significance to utilize the relationship between multi-temporal images for high accurate and efficient analysis and processing.Beside,thanks to the development of artificial intelligence,the combination of deep learning and remote sensing is becoming closer.However,in the current studies,researchers simply migrate it to remote sensing image processing,needing further improvement.How to improve the application of deep learning in remote sensing field based on the characteristics of remote sensing data and the use of high-performance learning tools is worth further study.Therefore,taking multi-temporal remote sensing images as input,this paper employs the correlation of different temporal images to carry out research on cloud detection and repair of multi-temporal remote sensing images,and use deep learning technology to study cloud detection of a single image so as to complete cloudy image processing workflow.The main works are as follows:(1)The cloud detection of optical remote sensing images based on semantic segmentation network is studied.The traditional machine learning methods usually have small model size,and the training relies on artificially designed features,which limit their feature representation and model learning ability.Methods based on convolutional neural network have larger model capacity and can directly encode high-dimensional features and learn model parameters in an end-to-end way,of which the performance is significantly better than traditional machine learning methods in tasks in computer vision field,such as image classification,target detection,semantic segmentation,and so on.Existing researches mainly focus on migrating advanced networks in computer vision to remote sensing image processing,using semantic segmentation methods to segment remote sensing images into cloud,cloud shadow,land-cover to achieve cloud detection,but ignoring the combination of characteristics and actual processing manner of real remote sensing images.Remote sensing images have much larger size than training samples of convolutional neural network.So in fact,they have to be cut into small image blocks before being fed to networks for inference.By adding a multi-label classification branch to the Ushaped semantic segmentation network,semantic category information of the image block can be identified and category-related feature representation can be learned as well.Then the feature representation is fed back into the decoder of semantic segmentation network,enhancing category-related features and weakening category independent features of the decoder selectively so as to improve the accuracy of semantic segmentation.(2)The cloud detection method of multi-temporal cloudy remote sensing images based on matrix decomposition is studied.The current cloud detection methods are mainly applied to a single remote sensing image.A few cloud detection methods for multitemporal images require a completely cloud-free image as reference.At present,there is no available methods that directly utilize the complementary nature of information between multi-temporal images for cloud detection.To adapt the growing demand for multitemporal image analysis and processing,and to better mine information in time dimension of remote sensing images,this paper takes multiple images covering the same area and imaged at different time as the research object,and apply matrix decomposition method to realize batch cloud detection.Arranging multi-temporal images into matrix format,the correlation between images is converted into the correlation between matrix columns.Using the matrix decomposition method,the observed matrix can be decomposed into a low-rank matrix,a group-sparse matrix and a noise-sparse matrix,corresponding to cloudless land-cover information,cloud and cloud shadow,and noise in multi-temporal images respectively.The final cloud detection result can be generated by directly binarizing the group-sparse matrix and rearranging to image format,without any post-processing.Different from plain sparse constraints,the group-sparse strategy utilizes the spatial continuous property of cloud pixels and constrains on superpixel targets,which can reduce noise in the final detection results effectively.In addition,the matrix decomposition method uses low-rank matrix to model the land-cover,its low-rank constraint is capable of combining with the two-dimensional geometric transformation model,which can handle non-pixel-level registration between multi-temporal images and expand the cloud detection ability.(3)The restoration method of multi-temporal cloudy remote sensing images based on matrix completion is studied.The current methods for cloudy images restoration are mainly for a single image,and rely on cloudless images as reference.A small number of cloud restoration methods for multi-temporal images essentially restore single images one by one,unable to fulfill batch processing simultaneously.In this paper,multi-temporal cloudy images are used as input,and the correlation between them is directly utilized to complete the restoration of all cloud pixels.Arranging multi-temporal images into matrix format and regarding pixels occluded by cloud and cloud shadow as missing information,low-rank matrix completion can be used to fulfill all missing information in batches.Considering the actual image restoration requirements,original low-rank matrix completion method is improved.The Frobenius norm is replaced by L1 norm to enhance robustness.To ensure original observation information reservation as much as possible while repairing invalid pixels,different weights are set for areas to be repaired and the original observation information.Besides,based on cloud masks,a certain range is extended to set the buffer zone,in which liner weights transited from invalid areas to valid areas are set in order to reduce the loss of the original observation information in the buffer zone while improve the smoothness of the restoration results.In summary,this paper implements the entire workflow of cloud detection and restoration for optical remote sensing images.In order to verify the effectiveness of the methods in this article,quantitative and qualitative experiments were conducted on the three proposed methods in the research contend sections.Finally,feasibility verification and limitation analysis of the overall technical solution on real large-scale remote sensing images are carried out as well.Extensive experiments have demonstrated the effectiveness of the proposed methods,providing a reference for cloud detection and restoration of multitemporal optical remote sensing images.