Research And Application Of Virtual Simulation Interactive System Based On Human Skeleton Information

Virtual Reality(VR)is a computer simulation technology that can transform a real environment or a conceived environment into a virtual scene.With the development of human-computer interaction technology,the research goal in this field has gradually shifted from the development of experimental functions to the improvements of user immersion experience.Previous virtual simulation experiment systems are interacted using mouse,keyboard,and VR handle.With the interaction approaches above,the degree of immersion is reduced in real applications.In order to increase the immersion experience of VR,this thesis develops a virtual simulation interactive system based on bone information.This system uses bone information collected by somatosensory equipment to recognize human actions since human actions are more natural if applied in human-computer interaction.In summary,this thesis conducts research from the following three aspects:(1)Study the acquisition of human bone information and the way to construct data optimization algorithms.Firstly,linkage between the somatosensory device “Kinect” and the Unity3D platform is researched.Also,how to obtain the coordinates of human joint points is studied.Secondly,a data optimization algorithm is proposed to increase the accuracy of recognition.Thirdly,for the jitter phenomenon in the data collected by the somatosensory equipment due to environmental factors such as light,a data smoothing method is proposed.Next,in order to overcome the differences in human body,a data normalization method is proposed.Finally,considering that the human body's movements are mainly the changes of the angles between the joints,a vectorization method of the joint coordinates in the human body coordinate system is proposed.(2)Study the recognition method of human actions based on bone information.Firstly,eight kinds of action commands with corresponding meanings are presented for recognition.These eight commands are divided into three categories: making a fist,waving hand,and stride.Secondly,the motion feature extraction is performed by considering the vectorized angle of the joint points as the recognition feature and discarding the joint points that do not contribute to the motion.The relative angles among the joints are added into the extraction method.Thirdly,as the dynamic time warping algorithm of human action matching algorithm has singularities and high time complexity,an improved method is proposed.Next,aiming at the singularity problem,this thesis adds the first-order differential and the second-order differential on the original data,which are weighted as the local distance.Finally,in order to reduce time complexity,this thesis makes improvements from three aspects: direct constraint method,data abstraction method,and distortion threshold.(3)Design and implement a virtual simulation system based on human bone information for simulated experiment teaching.First of all,an online multiplayer simulating experimental platform is created using Unity3 D to meet the needs of collaborative experiments among multiple persons.Secondly,functional modules such as human motion recognition,collaborative interactions,and virtual scenes are completed.Finally,the graphical user interface of this system is developed by C# and the 3D scene is added into this system.Human actions can be used as interactive approach in virtual scene.The designed system solves the problem of lack of interactivity and immersion in the previous virtual simulation system's interaction mode with mouse,keyboard,and handle.