Research On Design And Key Technology Of Mixed Reality System

With the continuous development of the computer field,mixed reality technology is more and more applied to various fields.Mixed Reality system is different from Virtual Reality system,which allows people to keep in touch with the real world and the virtual world.The most critical techniques can be divided into two categories.First,the fusion techniques in mixed reality need to make the virtual world and the real world of the perfect fusion.Fusion techniques include the illumination consistency and geometric consistency between real objects and virtual objects.Second,interactive techniques in mixed reality needs to make people communicate with the virtual world.Interactive techniques include speech recognition,target tracking,gesture recognition and so on.In addition,multi-target tracking in 3D space is also an important technology in mixed reality system.It can not only ensure the geometric consistency in fusion technology but also act as an interactive method to feedback the user's motion to the system.In this thesis,the design of mixed reality system and its key technical problems are studied.A framework of a mixed reality system is designed and implemented,which can solve the problem of fusion and interactive aspect of key technologies.A three-dimensional space multi-target tracking algorithm based on Marker is proposed to solve the most important and difficult geometric consistency problem in fusion technology.The algorithm can track and locate multiple similar targets in a certain space,without wearing a large number of complex and cumbersome external devices.It is easy to operate and it can achieve good real-time.The main contributions of this thesis are as following:1.The architecture of a multi-user interactive mixed reality system is designed and implemented which supports multiple users simultaneously interacting with the system.In the system,the real-time multi-user identity detection and 3D position tracking based on computer vision is implemented and the problem of illumination and geometric consistency in mixed reality system is solved.2.A multi-target recognition algorithm based on Marker encoding and decoding is implemented.Through capturing images from the camera and analyzing the information in the image,the algorithm can locate the markers from the images.The binary operation,contour extraction,polygon fitting,perspective transform,sampling,coding matching and pose estimation is used to determine the different position and of multiple markers and distinguish them.3.A multi-target recognition algorithm for 6-DOF tracking is purposed and implemented which solves the problem of the low recognition rate when the light changes significantly and unevenly.Three dynamic threshold methods are proposed to reduce the sensitivity of the light.A contour filter strategy is purposed to enhance the reliability of the system.At the stage of feature extraction,the median sampling method is implemented to enhance the reliability of the system.At the stage of coding,the Hamming code error detection mechanism and contour matching algorithm between adjacent frames is implemented to enhance the stability of the system.At the stage of localization,the pose estimation based on perspective matrix is implemented to enhance the accuracy of the markers.4.Experimental environment is designed and set up by using stereovision algorithm and multi-target recognition algorithm at 6-DOF to track and locate multiple targets in three dimensional spaces.Experiments are designed to verify the effectiveness of our algorithms.According to the experimental results and data,a detailed analysis and discussion are presented to prove that the accuracy and real-time performance of our algorithms.