Study On Virtual Camera System Based On Inertial Sensors And Unity3D

With the development of computer hardware and software technology, computer graphics and virtual reality technology in the past two decades, the virtual camera technology has entered a new era of development. Modern television programs must have the support of virtual camera technology. The use of virtual camera technology can be combined computer-generated virtual scenes with the real performer image, and showing a beautiful fantasy world to audience that they never experienced visual enjoyment. Now virtual camera technology has gradually extended to various fields from film and television production industry. It has a strong compatibility and the virtual scene can be generated in many 3D modeling making software. However, advanced virtual camera systems are based on workstations. They use the high-performance graphics processor to deal with video images and increase the cost of their development greatly, it is very hardly for some small-scale film and television production companies or local TV stations to pay the high costs.This thesis is based on MEMS sensors and Unity3 D software in order to develop a virtual camera system with low cost and simple operation. It mainly involves data acquisition, drift error compensation, video keying and Unity3 D technology. The main design principle of virtual camera system is that the presenter or actor's performance video taken with the real camera will be processed with keying technology and composited with the virtual image that generated from three-dimensional software in Unity3 D. The lens shift effect of composited video must be consistent with the real camera's lens parameters in order to avoid target prospects "float" on the virtual background. Virtual background images need to be created on computer or use the real image that shot ahead. Such background images can reuse and modify. In addition, it can reduce the production costs of television programs and improve the making efficiency and quality of films and TV production greatly.This thesis focuses on the data acquisition of the camera attitude data and video keying. The attitude data of real camera is transported to computer by MEMS sensors through the space tracking model based on the calculated acceleration integration. It need the Kalman Filtering to take error compensation to the data due to drift error of transfer process. After that, the data binds on the MainCamera of Unity3 D to achieve motion attitude synchronous of virtual and real camera. The shooting blue screen background video uses chroma key keying algorithms in GPU to accelerate the speed of keying and integrate into the object of Plane in Unity3 D by the way of map, and it can also be edited or previewed in real-time.