Research On Virtual View Rendering Method Based On Depth Image

Depth image based rendering methodis the most common method of virtual viewpoint rendering,generating arbitrary view video from other viewpoint video information.Virtual viewpoint rendering method is widely used in 3D products,and it is a very important technical process in the production chain of 3D product system.It is one of the important research topics in the field of computer image processing.The core principle of the method of depth image based rendering is three-dimensional Image Warping equation,that establishing pixel mapping relation between reference viewpoint image and virtual viewpoint image by the camera's internal and external parameters matrix of reference viewpoint and virtual viewpoint and depth map information of reference viewpoint.Disocclusion problem arises in the virtual viewpoint rendering that the regions covered by the foreground objects in the reference view may be visible in the virtual view,which is caused by the lack of image information.In the process of image fusion based on multi view merging,the selection of fusion method is also a problem to be solved.In this paper,image inpainting algorithm is used to fill hole and improved the algorithm.A new priority calculation method is proposed,using a weighted average of structure tensors as data item to improve the spatial local consistency and then improve the filling order;and also using local threshold segmentation to reduce the fill priority of the foreground pixels,that reducing the filling order of foreground pixels and avoiding the propagation of the foreground.In addition,a new formula to measure patch distance and a method to recognize the source space is proposed,using gradient and position relationship,the patch distance formula is improved in the search area to choose more appropriate matching patch to fill the target patch;In view of the disocclusion generated by the occlusion,use the depth image and the maximum between-cluster variance algorithm to recognize the relative background as the search field and reduce the mistake filling due to mismatch.The experimental results show that compared with the existing methods,the image quality of virtual viewpoint rendering has been greatly improved.In this paper,using the generalized regression neural network,an image fusion method based on machine learning is proposed.The neural network prediction model is established by using the data of the training samples,and use the image of to be fused as the test data to get the pixel value of the virtual viewpoint image.The experimental results show that the results of the proposed method are better than the traditional fixed weight image fusion methods.