Research On Interaction Technology Of Human Action And Gesture Recognition Oriented To Virtual Simulation Of Equipment Operation

Virtual simulation of equipment operations,as an important part of manufacturing simulation technology,plays a unique role in new product manufacturing,equipment operation inspection and operation training.Manufactur:ing virtual simulation is one of the virtual reality applications.Human-computer interaction is a key feature of virtual reality technology.In order to achieve a natural experience of human-computer interaction in virtual reality,special input devices need to be designed.The traditional interactive methods include keyboards,mice,three-dimensional space tracker,three-dimensional spatial positioning ball and data glove.These devices are complicated in operation and unnatural interactive experience.In order to solve the above problems,an interaction method of human posture gesture and gesture command recognition using optical visual equipment is proposed to improve the interaction and reduce the difficulty of interaction.The main contents of this paper include:The first chapter summarizes the domestic and international research and development of the interaction technology of interactive simulation of equipment homework and the human interaction and gesture recognition technology,expounds the research content and significance of this paper,and finally introduces the organizational framework of this paper.In the second chapter,a human behavior gesture recognition interaction based on the PCA-KNN algorithm is proposed.Acquisition of human skeletal data information,noise reduction of skeletal information data,elimination of body shape differences and normalization.PCA is used to extract the main components of the skeleton features,which are used as the input of the bone similarity calculation of the nearest neighbors of KNN to obtain the real-time human gestures category.The result category is used as the interactive trigger instruction.In addition,hand joint trajectory guidance interaction is added.The third chapter,an RGB-D gesture image recognition interaction technology based on HMM-SVM algorithm is proposed.The color image and depth image collected by Kinect are combined as input,the foreground gesture is segmented,the Hog and Sift feature of gesture image is extracted,and the model of Hidden Markov HMM and Support Vector Machine SVM is used to identify the real-time Gestures,get gestures instructions for interactive trigger.The fourth chapter realizes the follow-up motion of multi-chain joint virtual man and Kinect skeletal joint mapping.Firstly,the simplified Kinect skeletal joints are bound with 'the virtual human and a multi-joint chain rotation model is established.Secondly,according to the interactive requirements of virtual simulation,the action and gesture database is constructed,the corresponding instructions of mapping are identified,and the virtual human's instruction interaction is enriched.Using median filtering technology to solve the problem of dislocation of virtual human joints,then the follow-up movement is more stable.The fifth chapter develops a human body skeleton gesture and gesture recognition interaction system for device-working virtual simulation.The system mainly includes such modules as skeleton-driven virtual man,gesture recognition,gesture recognition and device simulation.The sixth chapter summarizes the main research contents and innovations of this dissertation,and looks forward to the current deficiencies and the follow-up directions that need further study.