Design Of Virtual Roaming System Based On Natural Human-computer Interaction Techniques

Abstract:Natural Human-Computer Interaction (HCI) is one of the most frontier and hot direction in the field of computer science today. Kinect contains many important HCI techniques, such as depth image, skeleton tracking, face tracking, speech recognition, etc, in the medical rehabilitation, virtual simulation, robot vision, entertainment,3d modeling, etc, has a broad prospect.The Paper is based on the Kinect HCI techniques, studied the in-situ walk recognition algorithm, Face-Mouse algorithm and Kinect application in virtual roaming. Firstly the Paper analyzed human in-situ gait, put forward the knee Angle recognition method, which is based on human skeleton joint coordinates and triangular geometry, eliminates individual height difference, improve the applicability of in-situ walk recognition. Secondly, this paper proposed a simple method of Face-Mouse, based on Kinect face tracking technology, calculated pitch and yaw Angle, transformed into screen coordinates, and then change coordinates of Mouse, to achieve the purpose of controlling Mouse with face or head. Thirdly, designed a virtual roaming system by Unity3D, triggered forward button by in-situ walk, adjusted angle of view by Face-Mouse, switched function modules by voice commands, and designed the wired and wireless hardware connections. All algorithms are substantiated by experiments.In this paper, the development of virtual roaming system, is a natural human-computer interaction technology applied to the actual project, achieved a high degree of freedom of computer operation, is a highly immersive and easier promotion technique of virtual reality.