Research On Hand Position Detection In Semi-virtual Reality Cockpit

Flight simulators are essential pilot training-and-checking equipments in aviation industry.Compared with traditional flight simulators, semi-virtual reality based flight simulator cockpits havegreat potentialities in the applications for its compact structure and relatively low price. In order toimprove the performance of semi-virtural reality cockpit, several key technologies of imageacquisition and image processing, especially for hand position detection of human-machineinteraction in virtual reality scenes are studied. This paper provides several methods to detect anddisplay the human hand in HMD (hand-mounted displayer). The contributions of this dissertation aresummarized as follows:Firstly, this dissertation investigates a special way to interact with virtual reality environment incockpit. because the semi-virtual reality cockpit use HMD to display the virtual evnironment, the pilotcan not see the real environment inside the cockpit，which needs to be generated in VR scene.Moreover, it is difficult to interact between human and machine. According to the specialties of virtualscene and the demand for interaction in semi-virtual reality cockpit, the hand tracking based on imageprocessing technology is constructed. By image acquisition and image processing, we make thecameras3D hand model in virtual environment correspond with real hand in real environment.Secondly, this dissertation investigates the color space and color stability. The color informationis an effective feather in image processing technology. The human hand or color marks are extractedfrom images by colors. However, the color feathers in the image will be unstability if the environmentlights’ luminous intensity and direction are changed. In order to improve the stability of colors, thisdissertation uses region grows methed with block algorithm to find the the optimal colorspace. Experiments also prove that changing the block size and color coverage in block could makethe color marks be more stable.Thirdly, color marks based binocular vision hand position detection method is proposed in thisdissertation. By pasting three different color marks on the back of hand and detecting the3D positionof color marks, the hand’s position and posture can be calculated. Considering the stability of colormarks, new decussate marks are used in this method. The point of intersection can be detected byHough Transformation Technology. The experimental results show that this new marks’ centroidindicative of stability.Fourthly, a naked hand position detection method by camera array is proposed in this dissertation.3D naked hand position detection is a crucial technology in virtual reality interactive system. Humanhand is a multiple degree of freedom soft body and has a wide range of motion. So human hand is hard to track and locate. This dissertation proposed a new method to tracking human hand by cameraarray based on palm segmentation to solve these problems. Firstly, we extract the contour of handbased color information in single camera. Secondly, we segment the palm from hand contour andlocate the palm in2D image. Finally, we use local spatial outlier detecting algorithm to select thevalid cameras in camera array and apply projective geometry to calculate the3D position of handobject. The experiment shows that this method has a strong anti-interference to different hand gesture.By using camera array this method can detect hand position in wide region and can get precise data byuncalibrated cameraFinally, this dissertation proposed a special way to composite images of hand in cockpit. Thehand gesture recognization is the further work In Semi-virtual Reality Cockpit System. However, theocclusion between hand and other objects would make the hand detection inaccuracy. Theself-occlusion of hand also exists in this environment. This dissertation proposes a dynamical lightfield method to deal with these problems. By setting the focus plane of the light field, the occlusionbetween hand and other objects problems can be solved. In order to track the hand palm plane andsolved the self-occlusion problem, we change the light field angle dynamically. Experimental resultsshow that this method can solve the occlusion problem in semi-virtual reality cockpit. The light fieldwhich is created based on the object hand can improve the efficiency of tracking system.