Research On 3D Light Field Rendering Algorithm Based On Deep Learning

In recent years,with the continuous development of 3D light field display technology,more and more 3D light field rendering algorithms have been proposed.For light field display devices based on multilayer screens,the parallax comes from the occlusion relationship of multi-viewpoint images on the multilayer light field display screen to achieve light field information reconstruction,which can effectively solve the conflict of focus and vergence adjustment in 3D display..The traditional multi-screen light field rendering algorithm mainly uses non-negative tensor decomposition to iteratively approximate the pixel data of each display layer to accurately reconstruct the features of different depths in the light field on its corresponding display layer.However,traditional algorithms have limitations.For example,the iterative algorithm takes too long to be suitable for display systems that require high real-time performance.The emergence of deep learning methods,especially the emergence and development of convolutional neural networks,provides solutions for real-time and wide-angle stereoscopic display.At present,the methods based on deep learning also have the following two problems: first,because there are not enough relevant datasets,the universality of the deep learning model is not strong enough,and the change of the scene has a great impact on the accuracy;second,the convergence speed and convergence effect of the deep learning model are not good enough.In addition,whether it is a traditional iterative algorithm or a deep learning algorithm,the field of view of light field rendering and reconstruction is relatively small.Therefore,in view of the advantages and disadvantages of the current light field rendering algorithm,this paper mainly focuses on the research on the 3D light field rendering algorithm.The main research contents and contributions are summarized as follows:1.Based on the iterative algorithm,the light field rendering algorithm to further improve the field of view angle is studied,mainly including the principle of non-negative tensor decomposition and the basic principle of the analytical method.This paper firstly parameterizes the light field data and display layer data.It is converted into a form suitable for analytical processing,and is calculated and verified on the light field with only horizontal parallax and the light field with horizontal and vertical parallax respectively,initialized by the pixel data of the display layer,and passed to the pixel data of the display layer.The multiplication rule is iteratively updated and experiments are performed under different scenes and parallaxes.The experimental results show that the algorithm can achieve better rendering and reconstruction of the 3D light field with a large field of view angle.2.A large field of view angle light field rendering algorithm based on general convolutional neural networks is proposed.First,the feasibility of applying convolutional neural networks to light field rendering is analyzed,and then the light field data and the data of the display layer are parameterized,which is transformed into an optimization problem to solve.Second,a variety of different convolutional neural network structures are designed,and a suitable training set is designed,and dense data augmentation is carried out,so that there is enough light field data to train the model.Afterwards,they are trained and tested on the light field with only horizontal parallax and the light field with horizontal and vertical parallax,respectively,through different scenes,different parallaxes and different display layers and other conditions.The models’ robustness and generalizability are tested.The experimental results show that the light field rendering of comparable quality can be achieved by reducing the training parameters,and the convolutional neural network algorithm can improve the processing speed to achieve real-time performance for all test scenarios.At the same time,the quality of rendering effect is comparable to traditional algorithms such as iterative algorithms.Therefore,the convolutional neural network algorithm has strong robustness,real-time and universality.3.A large field of view angle light field rendering algorithm based on residual network is proposed.First,the limitations of general convolutional neural networks are analyzed,and then the advantages of residual networks and the feasibility of applying them to light field rendering and reconstruction are analyzed.Several different residual networks are designed,trained and tested on light fields with only horizontal disparity and light fields with both horizontal and vertical disparity.The experimental results show that compared with the general convolutional neural network algorithm,the residual network algorithm can achieve faster convergence speed and shorter training time without increasing the training data,and improve the efficiency of data training.The rendering reconstruction accuracy of the scene’s light field is comparable.Through the above research,a large field of view angle light field rendering algorithm based on deep learning has been realized.On the premise of ensuring robustness,real-time and universality,it has achieved better rendering and reconstruction effects and improved model training.convergence speed.This makes it possible to produce large-viewing-angle light-field displays based on multilayer screens.