Research Of Human-Computer Interaction In Immersive Virtual Shooting Training System

Virtual reality system uses computer technology to generate realistic virtual environment, providing the simulation of visual, auditory, tactile and other sensations of human being. It allows the user to have a sense of immersion. It has important application value in entertainment, education, military training and many other areas, so it's interactive mode become a research focus in the field of computer in recent years. Data gloves, data suits and other sensors that are common used can provide humanized interaction, but they are expensive and make the user discomfort.This thesis is based on the research and development of an immersive system for shooting training. A highly immersive system environment has been designed and established. The related computer vision technology is used to realize the natural human computer interaction and real-time system response. The main contributions of this thesis are as follows:1) Design the interactive hardware platform of the immersive training system, set up a wide-angle projection display device letting the user have a high degree of battlefield immersion in vision, design a wireless imitation gun with infrared laser making the user shoot by a natural way same as the real shooting action, use the method of cameras with infrared lens capturing the infrared reflected light accurately to calculate the 3D position and motion of the user in real-time, ensure the immersive scene produce realistic change with the displacement of the user's body.2) Capture the shooting hit point on the projection screen at the moment that the user pull the trigger of the imitation gun accurately and in real-time, use the 2D projective transform together with the corner detection to rectify the deformation of the projection screen in image due to camera positions, then use threshold segmentation and affine transform to calculate the coordinates of the shooting hit point on the projection screen, and finally correct the coordinates to get more accurate result.3) Study the camera imaging model and camera calibration principle, use the Zhang zhengyou's method to calibrate the two cameras that used to capture the human body, thereby establishing the relationship between the virtual scene and the real 3D space.4) Capture the motion of the user's body accurately and in real-time, use the background subtraction and image morphological operations to extract the silhouette image of the human body, use the apar stripe detecting all body parts to identify the marks on the human body, and then use the Kalman filter, epipolar restriction and human constraints to track the marks in the video frame sequences, meanwhile, use the method of binocular stereo vision calculating the 3D positions of all the marks to achieve the popurse of capturing human motion accurately and in real time.