Research And Implementation Of Panoramic Human-Computer Interaction System Based On Kinect

With the vigorous development of information technology,computer plays an indispensable role as an important tool in people's daily work,life,learning,education and entertainment.At the same time,the way of human-computer interaction is also changing gradually,from traditional peripheral devices such as mouse,keyboard and handle to touch-control operation based on touch panel.However,human-computer interaction still belongs to contact mode,which can not be separated from the restriction,complexity and limitation of hand-control.With the emergence of various new technologies,such as super-large display screen,AR and virtual reality,human-computer interaction based on computer vision has received extensive attention and research,The human body information and gesture information in human activities are captured by camera,and then analyzed and processed by computer to recognize the meanings of various gestures and gestures,so as to realize more natural,intuitive and computer-based human-computer interaction.Simple interaction.Through Microsoft's Kinect somatosensory device,it can efficiently and cheaply acquire real-time motion information such as human color images,depth images and skeletal data,Hmaking human-computer interaction more simple and diversified,In this paper,Kinect sensor is used to acquire real-time human motion information,to study gesture recognition and human motion recognition,and a set of human-computer interaction system is implemented based on the research results.The main work of this paper is as follows:(1)A static gesture recognition algorithm based on fingertip detection with average curvature and palm distance is proposed.Through the color image,depth image and skeleton tracking technology provided by depth sensor Kinect v2,the tracking,location,detection,segmentation and noise reduction of gesture area are realized.Then the palm center of gravity and the weighted Hough circle are used to locate the palm center more accurately.Finally,the palm region is sampled and fingertips are detected by mean K curvature and palm distance to recognize five commonly used gestures.(2)A new method of human motion recognition based on key frames and skeletal information is proposed.The depth sensor Kinect V2 is used to obtain the real-time information of human skeleton points,and the weighted K-means algorithm is used to extract the key frames in human motion video.Then the human body structure vector is constructed by using the three-dimensional coordinates of 25 skeletal points in each key frame,and the joint angle and the vector modulus ratio are calculated.Finally,the similarity between the feature sequence of the key frame and the action template in the template library is calculated by the optimized DTW algorithm,so as to recognize the real-time action of human body.(3)Design and implement a new human-computer interaction system based on Kinect sensor in panoramic scene.By combining the above gesture recognition algorithm with human motion recognition algorithm,a set of human-computer interaction system is designed for command operation of different gestures and limb movements,which is applied in the immersion roaming of Huizhou merchants'mansion under the Huizhou cultural display and virtual scene.The system runs smoothly and naturally,operates simply and efficiently,and has a good user experience.At the same time,the validity of this method is verified,and it has practical popularization.