| With the advent of new technologies such as human-computer interaction,virtual reality,augmented reality,and the Internet of Things,traditional mice and keyboards cannot meet the interactive needs in these situations.The application environment characteristics of these new technologies are more inclined to the user's natural interaction with the system using simple and efficient.At present,human-machine natural interaction technology mainly has key technical problems such as efficient natural interaction,user movement and navigation in the scene,and multi-modal cooperative interaction.Recognition and interaction of gestures,gestures,faces,etc.based on computer vision is a research hotspot of natural interaction technology.This paper proposes a gesture recognition technology based on deep neural network to solve the problem of natural gesture interaction;The face recognition technology based on HOG and SVM algorithm and the gesture detection based on Kinect are proposed to solve the problem of natural movement and navigation;In the development environment of Unity3 D and C#,the natural interaction system in vi rtual reality environment is designed and implemented;Finally,the application of the virtual space configuration demonstrates the technical solution proposed in this paper.This article specifically includes the following four aspects of work.(1)Using Kinect Studio to record gestures and pose samples,the collected samples are trained in Visual Gesture Builder(VGB)to complete the detection of gestures and gestures and apply them to the interaction and navigation of the virtual environment;Based on the deep neural network,the object detection API is used to complete the gesture training in Pycharm,and then the detection result is applied to the natural interaction with the furniture in the virtual space configuration,which improves the accuracy of gesture recognition;The Kinect Studio and VGB training results are combined with the deep neural network to complete the gesture detection scheme,and the recognition rate of three gesture recognition schemes is analyzed.(2)The human skeleton data acquired by the Kinect device defines the semantics of gestures and gestures(eg,stepping,waving,etc.);The face recognition is implemented by using HOG extraction feature SVM training,and the HOG extraction feature process is optimized by the integral graph method,which is applied to the navigation of the perspective in the virtual environment;The face detection of Haar and LBP algorithm was completed by using opencv_haartraining.exe and opencv_traincascade.exe,and the recognition rate,training time and detection time of the three schemes were compared.(3)The hardware environment of the combination of splicing screen,Kinect and camera equipment was constructed,and the application placement of the equipment was analyzed.The software environment is the main plug-ins for Unity3 D and MS-SDK,ML-Agents and TFSharpPlugin.The deep neural network training environment is Pycharm and Object Dection API.(4)The roaming and configuration of the virtual space is completed based on the implementation of the Unity3 D engine and the above natural interaction technology.The natural interaction of virtual reality environment is a hot issue at present.This paper studies the natural interaction technology in virtual environment and applies it to the space configuration system based on natural interaction of virtual environment.Through Kinect and ordinary camera equipment,the dynamic scene behavior control is completed by gesture,face and posture detection,and the effect display is realized on the splicing screen,and the dynamic interaction with the room can be experienced in real time.For example: the rotation of furniture,moving and watching the scene of the room,etc.Users can choose home appliances and home configurations on this platform to mix and match as desired.The system's roaming and interaction effects can best meet the needs of users familiar with the home layout in advance. |
PDF Full Text Request