Research On Video Conference System Basedon Mixed Reality

With the development of computer graphics,sensor technology,human-computer interaction technology and other technical disciplines,Mixed Reality technology is becoming more and more perfect,and gradually enters people's daily life from science fiction movies.At the same time,the development of communication technology also indicates that people's demand for information diversity is increasing,and Mixed Reality technology provides this possibility.The traditional video conferencing system has the disadvantages of single information,lack of interaction means,and lack of immersion.The form and function of the video conferencing system based on the Mixed Reality technology can make up for the deficiencies of the traditional video conference to a certain extent,which is of great significance.The paper mainly focuses on the design and implementation of a video conferencing system based on Mixed Reality.Several components of a video conferencing system are discussed,namely,data collection,data transmission,display and interaction.The main research contents of this paper are as follows:(1)Discuss how to obtain 3D data of the human body through a depth camera,and then reconstruct the human body in 3D.For Mixed Reality-based applications,the biggest feature is to give the user an immersive 3D scene,so providing 3D information is the basic function of the application.The consumer-grade depth camera can acquire the surrounding three-dimensional information at a higher rate.Based on the principle and structure of the depth camera,this paper calibrates the depth camera to obtain the internal and external parameters of the camera,and then denoises and fills the acquired depth map.After the high-quality depth map,the 3D point cloud can be recovered based on the pinhole camera model,and the 3D mesh model of the human body with texture is finally obtained by the fast meshing algorithm for the ordered point cloud.(2)The application of traditional video codec technology in 3D data transmission is discussed.Firstly,starting from the characteristics of the depth map,and then through the contrast experiment,the codec of the three video codec standards supporting H.264,HEVC and VP8 is used to encode and decode the depth map and the depth map sequence.Finally,the experimental results are compared and the experimental results are analyzed and evaluated,which lays a foundation for the transmission part of the video conferencing system based on Mixed Reality.(3)Discussed various ways of human-computer interaction in a Mixed Reality environment.Firstly,introducing the interactive tools commonly used in virtual reality systems.At the same time,three human-computer interaction schemes are designed according to the hardware characteristics,and then multiple interactive tasks are set based on the video conference usage scenarios.Finally,the experimental results are obtained through comparative experiments and questionnaires.For the experimental results,the advantages and disadvantages of each interaction scheme are evaluated from the perspective of usability.(4)Design and implement a video conference system based on Mixed Reality.Firstly,the system is analyzed for requirements,and the requirements are put forward from the two aspects of performance and function.Then the system framework and system flow are designed,the whole system is modularized and implemented one by one,and finally the whole system is tested and evaluated.