Research And Design Of The Integration Of Virtual Scene And Natural Interaction Technology

The fusion of virtual scenes and natural interactions is a comprehensive multidisciplinary new technology research field,which is widely used in medical,education and entertainment industries.Its research mainly focuses on two aspects: on the one hand,using computer vision and body classification methods;On the other hand,it studies body posture recognition methods and anti-interference methods in virtual scene design.Currently,the main problem is the efficiency of natural interaction and the fusion of virtual scenes.This thesis proposes gesture and pose recognition methods based on machine learning and spatial geometric algorithm matching to solve the problem of natural interaction efficiency;a method based on the combination of color image data and depth data is proposed to solve the problem of human body and virtual scene fusion;finally,under the development environment of virtual reality software Unity3 D and depth sensor Kinect,a virtual scene and natural interaction fusion system is designed and implemented,which verifies the feasibility and effectiveness of the technical solution in this thesis.This paper includes the following four main tasks.(1)According to the functional requirements of the virtual scene natural interaction system,typical case development of three-dimensional models and two-dimensional scene elements in the virtual scene is realized.Use 3DMax software to build bird and butterfly models,establish skeletal parent-child relationships,and use skinning technology to complete the flapping flight design;Import Unity3 D to bind the development script of the model,design the intelligent flight algorithm based on AI,and realize the natural group flight effect of birds.Created three-dimensional virtual scene models such as the sun and torches,and realized dynamic effects such as collision detection and gravity effects,and completed the construction of a natural interactive system development environment for virtual scenes.(2)The combination of color image data and depth data is used to realize the fusion of human body and virtual scene in natural environment,which makes up for the limitation of common green screen background.Aiming at the problem of poor edge segmentation effect of human body depth image after filtering and noise reduction,a noise reduction method combining mean filtering and Side Window Filtering algorithm is proposed to avoid the damage of image edge information,combining binarization and morphological gradient operation to enhance the edge segmentation effect.Through the conversion of the depth image coordinate system and the color image coordinate system,the human body is integrated into the virtual scenes.(3)Use Kinect Studio to collect sample data containing gestures and postures,and perform machine learning training in Visual Gesture Builder(VGB)to complete the recognition of custom dynamic gestures such as sliding,grabbing,and moving;the spatial geometric algorithm based on bone point data is matched to complete custom dynamic gesture and gesture recognition;test and analyze the recognition rate under different background and lighting conditions.(4)Design a virtual scene natural interaction system based on Unity3 D and Kinect to test the effectiveness and feasibility of the solution in this thesis,it mainly includes two-dimensional and three-dimensional virtual scene interaction,natural gesture recognition,and integration of human body and virtual environment.The real human body is integrated into the virtual scene,adding logic rules to realize the collision interaction of the hand and the two-dimensional elements,and taking the torch,butterfly,etc.as an example to test the natural interaction functions such as grabbing,lowering,and moving.The fusion of virtual scenes and natural interactions is one of the current hot research fields.The real-time interaction between the human body and the virtual environment in the real environment has greatly increased the user's sense of reality,experience and interest.The research in this thesis provides a kind of ideas,methods and examples for related design.