Real Time3D Reconstruction Based Tele-presence Technology

Communication is an eternal human topic, and among the many waysto communicate face-to-face communication is the most effective one.However, as the economic globalization of the world face-to-facecommunication might not be always available or cost too much. It iseagerly desired of such a new multi-media communication technology thatbrings people at difference sites with immersive experience of face-to-facelike videomeeting. Thus Tele-Presence which integrates the virtual realitytechnology and communication technology has been a hot research topic inrecent years. Currently available solutions such as common videoconference softwares and commercial TelePresence systems provide only2D video conference experience and their display results never change withthe user’s viewpoint, which causes the loss of immersive feeling.Among these works in TelePresence research, the multiple RGB-Dcameras based real-time3D reconstruction algorithms show excellentpotential with great reconstruction quality and real-time framerate. In thispaper, we research and implement a real-time3D reconstruction basedone-to-one TelePresence system with multiple RGB-D camera whichprovides viewpoint-dependent display and preserves eye contactpartly.Specifically, the main contribution of this work includes:Firstly, this paper proposes a flexible TelePresence framework thatdivides the system into several independent modules:reconstructor module,tracker module, viewer module, coder module and network module.Secondly, this paper focuses on a multiple RGB-D cameras basedreal-time3D reconstruction algorithm. We improve the reconstructionquality with color compensation algorithm on color image and two-pass median filter algorithm on depth image while maintaining the real-timeframerate.Thirdly, we propose an image warping based approach that rendersthe viewing result under the new viewpoint locally with existing color anddepth images. This approach saves the network round trip and imageencoding time which reduces the user interaction delay.Last but not least, a serial of experiments are performed to show thesystem’s overall quality and performance. The experiment result proves thethe algorithms’ effectiveness in improving reconstruction quality andreducing the interaction delay.