Study On Key Techniques Of Multi-view Video Super-Resolution

The huge data size of multi-view video limits its application.Although mixed-resolution(MR)compression can reduce the data size of multi-view video,it causes the degraded video quality.Thus,super-resolution(SR)algorithms are needed in the receiver end to increase the video quality of low-resolution(LR)views of MR videos.SR algorithms of multi-view mixed-resolution(MVMR)video refines the quality of the LR images by utilizing high-frequency(HF)information from high-resolution(HR)images and can be divided into two stages,i.e.,HF information extraction stage and HF information merging stage.HF information extraction stage extracts HF information from HR views of MVMR video which contains view projection,coordinate check and projected image forming.HF information merging stage fuses HF information from multiple HR views and forms the HF parts of LR images.According to the relying spaces,SR can be divided into transform-domain and space-domain based methods.Space-domain methods process image pixels directly while transform-domain methods first compute transform-domain coefficients of image patches,then process the derived coefficients,and finally obtain the recovered image patches via the inverse transformation of the processed coefficients.Current transform-domain based SR only exploits the spatial redundancy of multi-view video and neglects its temporal redundancy.And in space domain,combining the local and non-local characteristics of images can help to improve the performance of SR algorithms.Thus,this thesis proposes two SR methods from the two aspects.Firstly,this thesis improves transform-domain(discrete cosine transform)based SR.The proposed transform-domain method simultaneously utilizes the spatial and temporal redundancy of multi-view video.To fuse HF information from different HR frames and views,this thesis designs global weight based on image correlation and local weight based on mean square error of patches.Secondly,this thesis proposes a new space-domain SR combining kernel regression(KR)and non-local means(NLM).KR is based on the local continuity of images.This thesis uses KR in the projected image forming operation to interpolate integer pixels from non-integer pixels derived from view projection.NLM,based on the non-local self-similarity,is used for HF information merging in this thesis.This algorithm can,on the one hand,alleviate the hole and crack artifacts of projected images and,on the other hand,reject dissimilar pixels in the similarity comparison stage of NLM and guarantee the accuracy of recovered HF information.This thesis compares the two proposed methods,finding that the space-domain SR works out better.Moreover,this thesis compares the proposed space-domain method with other SR method and the results show that the proposed space-domain SR outperforms them.