Research Of Pan-sharpening Via Neural Network And Deep Gradient-based Prior

Pan-sharpening is a domain-specific task of satellite imagery fusion,which fuses high spatial resolution panchromatic image with corresponding multispectral image,to enhance the spatial information and preserve the spectral detail of multispectral image.Given the amount of observation data is less than the unknown desired data,Pansharpening is an ill-posed inverse problem,while most existing traditional Pansharpening algorithms fuse multispectral image and panchromatic image in linear manners,which greatly restricts the fusion accuracy.Deep learning theory offers solution to cope with the ill-posed inverse problem of Pan-sharpening.Two algorithms which take advantage of nonlinear mapping ability of convolution neural network in different manners are proposed in this paper for Pan-sharpening.In this paper,the following aspects are studied:1).The employ of end-to-end network in Pan-sharpening is conducted,In this paper,spectral preservation and spatial enhancement are regarded as main research tasks like other works,generalization ability and feed-forward computation time are also taken into consideration,and employed as guiding concepts of models in this paper to improve the overall performance.During the network design,dilated multilevel block is proposed according to the experiment results which is intended to improve fusion performance and decrease the influence of shallow network architecture.For the generalization evaluation of this model,different types of satellite data is employed for test.2).The effect of spatial enhancement in gradient domain is researched,a gradientbased spatial enhancement term is proposed for Pan-sharpening,and optimized with the alternating direction method of multipliers.3).This paper researches and initiates an attempt to incorporate deep learning with model-based optimization for Pan-sharpening.The aforementioned optimization model is regarded as basic model,a gradient-bsed deep prior is proposed in place of spatial enhancement term in basic model,which breaks the linear restriction of it.As the deep network is trained in gradient domain,which allows for bypassing overfitting caused by network training in pixel domain,therefore,deep network can be adopted for performance improvement while maintain the generalization ability.