Research On Intelligent Virtual-real Interaction Technology Based On Gesture Recognition

The immersive virtual real interaction based on gesture recognition is an important research content in the fields of virtual reality technology,computer vision and computer graphics.Gesture interaction is the main human-computer interaction mode of non-contact language.The combination of gesture recognition and virtual reality simulation technology promotes the development of human-computer interaction technology and has a good application prospect.There is a growing demand for the friendliness of virtual reality simulation interaction.How to improve the accuracy of real-time gesture recognition and enhance the intelligence of virtual and real interaction is an important factor affecting its application and promotion.This article focuses on the Key Research&Development Plan Project of Shandong Province "Research of Common-Used Sign Language Words Recognition Technique based on Deep Learning"(number: 2017GGX10127),under the support of driving scenarios for simulation,the research based on gesture recognition nakedness of intelligent interactive technology,and design to achieve the gesture recognition method based on skeleton movement and gesture recognition method based on the deep learning,the user and virtual scene real-time interaction through natural gestures.Main research contents include:(1)A weighted DTW algorithm is proposed to analyze the extracted skeleton information,calculate the contribution degree and motion trajectory of each human skeleton joint,improve the DTW algorithm,and realize real-time coarse-grained recognition of continuous gestures;(2)ROI was selected from human hand images based on the location and distance information of human bone joint points,and feature extraction of ROI information was carried out using seven-layer CNN network to obtain fine-grained recognition of hand behaviors.(3)The vehicle driving simulation system based on gesture recognition was designed and realized.Unity3 D was used to simulate the vehicle driving scene.Kinect and HTC VIVE interactive devices were used to make the virtual environment interact with the user's gestures in real time.