The Research Of Light Field Technique Of Dynamic Object On Mobile Device

With the development of multimedia techniques,video techniques need to transform from two dimension(2D) to three dimension(3D).Compared with 2D video,the depth information of scene has been added to 3D video,and the reality of visuality has also been enhanced,which can provide the feeling of immension in all domains.3D video technology can be widely used in digital television,mobile device systems,3D video-conferencing systems,virtual reality systems,and so on.Among them,Mobile device systems which are more concerned about people.Mobile devices have gradually become an important way to information acquisition,processing treatment,person entertainment,and so on.Thus people hope urgently that they can enjoy the photorealistic rendering on mobile devices,for example,realistic display 3DTV,virtual reality roaming,etc.As a result of the use of battery-powered,limited memory space and smaller bandwidth on mobile devices,the photorealistic rendering system of mobile device should have the characteristics of low power consumption and low bandwidth. Traditional computer graphics is built on 3D mesh model rendering,the ultimate rendering effect is directly affected by 3D mesh model which whether can be built accurately.The entire rendering process is related with the complexity of scene, leading to the problem of large power.As the IBR has the character of no relation with the complexity of scene and the strong realistic,tt has become research focus on photorealistic rendering in the past decade.But due to the large data amount,it is very disadvantage for mobile devices with limited storage space and bandwidth.For these problems,this paper proposes a photo-realistic rendering technique of 3D dynamic object on mobile devices.The main work and innovation are as follows:1) For the character of small storage space and limited bandwidth on mobile device,this paper propose a light field rendering codec design of 3D dynamic object on mobile device.As the 3D mesh model from the image,it can decrease the complexity of artificial modeling.At the same time,with the geometric information of objects,the number requirement for the reconstruction of the target scene images can be a significant reduction,which has an advantageous of low-bandwidth data transfer for mobile devices.The experiment results show that the amount of data storage and data transmission have been reduced effectively,compared to IBR.2)Combined with the reconstructed 3D mesh model,this paper proposes a surface light field sampling methods on mobile device,analyse the issue of visibility before light field sampling,as well as the generation of the virtual viewpoint in detail.This method not only can achieve realistic results,but also the realization of free viewpoint rendering.The experiment results show that this sampling method records the light information of the object accurately,including the light intensity and light direction.3)As the light field sampling matrix contains a larger amount of data,it is disadvantage for the transmission of video data.This paper proposes an adaptive algorithm for the singular value decomposition,which can automaticly control the matrix decomposition level,guarantee the quality of the reconstruction while improve compression ratio.The surface texture and view texture can be obtained after light field matrix decomposition.We use vector quantization and S3TC compression algorithms to reduce the external and internal redundancy of texture images.The experiment results show that the amount of data has been decreased effectively under the premise of realism.Furthermore,S3TC is the standard texture compression algorithm which is widely support by graphics hardware,thus it is advantage for the hardware acceleration of realizing light field reconstruction.4)This paper proposes a hardware architecture design of light field rendering decoder facing on mobile device.We use self-designed graphics processor to accelerate light field decoding and realize the real-time rendering.At the same time, for the core module which is responsible for light field reconstruction—texture mapping unit,this paper proposes an FPGA design.In the implementation,there is an optimized hardware design of texture Cache through utilizing accessing methods based on 4×4 compressed texel block.The above optimization makes full use of the relationship between pixels,so the rate of Cache shooting becomes higher and the demand of bus bandwidth becomes lower.The experiment results show that this architecture can realize the realistic rendering of 3D dynamic object,present evidence of feasibility of 3D dynamic object realistic rendering on mobile device.