Research On The Algorithm Of Super Resolution Image Reconstruction Based On Direction Judgment

Super-resolution image reconstruction aims at improving quality of image and videos with the help of software.Thus,it meets visual perception and information discrimination requirements of people.Widely applied in the area of digital image processing,such as high definition video,medical image analysis,remote sensing detection,target recognition and video surveillance,this technology is blessed with broad market prospect and scientific research value.Since super-resolution image reconstruction is complex,it is affected by many factors,such as observation environment,noise,observation angle and movement etc.As a result,the outcome is uncertain and the research faces many challenges and difficulties,that's the reason why super-resolution image reconstruction is one of the research hotspots in image processing field.Some basic theories such as image reconstruction,super-wavelet transform,fractal geometry and quality evaluation of reconstructed image were introduced in the thesis.The problems of super-resolution image reconstruction were studied deeply and systematically in direction determination,and the preliminary exploration was made on its hardware implementation.The author aimed at continuously improving the clarity and fluency of the reconstructed image or video sequence by exploring the application of signal processing and new computer technology in the determination of motion and texture direction.Thus,people will be provided with better viewing experience and image information.Finally,a new reconstruction algorithm was designed and its hardware implementation was explored considering the difficult problems and practical needs of the traditional super-resolution image reconstruction technology.The research contents include(1)temporal super-resolution image reconstruction with improved determination criterion of motion direction based on Bandelet transform and fractal geometry;(2)temporal super-resolution image reconstruction with improved determination strategy of motion direction based on unidirectional and bidirectional fusion;(3)single-frame spatial super-resolution image reconstruction;(4)multi-frame spatial super-resolution image reconstruction;(5)hardware implementation framework of temporal super-resolution image reconstruction algorithm.The main achievements are summarized as follows:(1)Some new solutions were put forward according to the problems such as the motion estimation error,hole and overlap,occlusion and exposure,and local optimum etc.These problems arise as temporal super-resolution image reconstruction based on block matching motion estimation is affected by the determination criterion and determination strategy of motion direction.Firstly,two determination criteria of motion direction based on double difference combination cost function were proposed by referring to the calculation idea of SAD(sum of absolution difference)and they were applied to temporal super-resolution image reconstruction,taking advantage of the superior performance of Bandelet transform on multi-direction skill and the scale invariance of fractal geometry for image dimension description to represent the texture features of image blocks.On one hand,the SAD value based on image block pixel is kept in cost function of the motion direction determination criterion.On the other hand,the SAD value based on geometric flow direction or fractal dimension of image block is added.The accuracy and robustness of motion estimation results are improved.Secondly,a motion estimation strategy based on global unidirectional determination and local bidirectional fast determination was proposed.Besides,it was also applied to temporal super-resolution image reconstruction.The proposed motion direction determination strategy achieves the advantages of unidirectional motion estimation and bidirectional motion estimation.What's more,occlusion mask and exposed mask are carefully designed and used to mark the occlusion and exposed areas of the interpolated frame image.The problems of occlusion and exposure of reconstructed image,which is caused by the motion direction with bidirectional determination strategy,is solved by using the proposed masks.Experimental results showed that subjective and objective quality evaluation index of the reconstructed images through the temporal super-resolution algorithm with the proposed motion direction determination strategy have both seen obvious improvements.Finally,based on the powerful computing power of FPGA and its efficient support for its secondary development,the hardware realization of the proposed temporal super-resolution image reconstruction algorithm was explored.In hardware architecture,multi-stage sub-algorithms were obtained with split steps in the whole algorithm.Then,learning from the streamlined production in industry,the sub-algorithms are initiated step by step and processed in parallel.Thus,the reconstruction period of the super-resolution image is greatly shortened.Additionally,a RAM distribution structure based on data streamline loading was designed so that the demand for bandwidth of DDR is reduced.Tests on screen showed that the hardware architecture is helpful to realize practical application of complex temporal super-resolution image reconstruction algorithm.Of course,the parallel execution framework of sub-algorithms with RAM data streamline loading can also be applied to the hardware implementation of complex spatial super-resolution image reconstruction algorithm.(2)Edge sawtooth effect and contour blur problem are the key difficult problems of research on single-frame spatial super-resolution image reconstruction,especially on texture complex images.With Bandelet transform and total variation in wavelet transform domain,single-frame spatial super-resolution image reconstruction algorithm based on direction estimation of image geometric flow was put forward.To improve the performance of protection of image edges and textures,at first,a direction estimation of geometric flow in Bandelet block is decided with Bandelet transform.Secondly,Interpolation of one-dimensional signals for two-dimensional images in the direction of geometric flow is realized with regard to the second generation Bandelet transform.Thus,the spatial super-resolution image is preliminary reconstructed.In order to eliminate possible Bandelet block effect,total variation processing of the reconstructed image is conducted in wavelet transform domain.Finally,the edge of the reconstructed image is effectively protected,and the performance of the proposed algorithm is improved as well.(3)Because multi-frame spatial super-resolution image reconstruction is influenced by the following factors such as the change of observation angle,the number of observed images,and the difficulty to distinguish high-frequency information of the edge from noise,the result of image reconstruction is uncertain.Therefore,multi-frame blind spatial super-resolution image reconstruction algorithm based on learning and image orientation was put forward with the combination of Contourlet transform and artificial neural network.Firstly,an initial image with high spatial resolution is reconstructed by the method of traditional single-frame spatial super-resolution.Then,Contourlet bandpass directional subbands of the estimated residual of high spatial resolution image are generated by artificial neural network.Thirdly,the estimated residual sequence of high spatial resolution image is obtained by making inverse Contourlet transform to these bandpass directional subbands and the estimated residual sequence of high spatial resolution images.Finally,with the proposed adaptive weighted coefficient vector,it compensates for the initial high spatial resolution image reconstruction based on the estimated residual sequence of high spatial resolution image.Thus,the final high spatial resolution image is obtained.As to the proposed spatial super-resolution image reconstruction algorithm,there is no need to consider image registration and noise.Besides,taking full advantage of multi-directional and anisotropic properties of Contourlet transform,the edge of the image is effectively protected.The research results show that the proposed algorithm has good image reconstruction effect.