Design Of Multi-mode Sensing And Interactive Data Glove System

With the rapid development of computer technology level,computer equipment in people’s life to a great extent,the virtual reality fusion technology combined with all aspects of discipline knowledge emerged at the historic moment,and gradually widely used in education,medicine,military and other fields.In order to match the use of virtual reality technology,it often needs the assistance of cameras,operators and other equipment.On the one hand,it limits the user’s use space,and on the other hand,its defined operation mode is different from the real scene to some extent,which reduces the user’s immersive experience.Good man-machine interaction has become one of the keys in the use of virtual reality fusion technology.As the most flexible organ in human body,hand gradually occupies a certain position in humancomputer interaction,among which the human-computer interaction based on intelligent data glove is the most natural and immersive way,which has also become a research hotspot in recent years.However,most of the existing data gloves in the current market are not comfortable enough to wear,high cost to use and other deficiencies,and the application of touch,an important human perception channel,is not common,which reduces the immersion in human-computer interaction to a certain extent.By analyzing the research work at home and abroad,this paper summarizes the advantages and shortcomings of data gloves currently used in virtual reality interaction,such as the small number of acquisition channels,points out the importance of integrating haptic channels to improve human-computer interaction experience,and identifies the research objectives and established requirements of the multi-modal sensory interactive data glove system proposed in this paper.The data glove system consists of two parts: the hardware system of the lower computer and the software program of the upper computer.The lower computer system is responsible for collecting various hand information of the user,including hand temperature and humidity information,pressure perception information,finger curvature information and palm posture information.The upper computer software program is responsible for data submission and monitoring of the information obtained by the data glove and real-time reappearance of the virtual scene to realize the integration between the virtual and the real.Bluetooth is used for real-time communication between the upper computer and the lower computer to reduce the limitation of data glove space.In the design process,for the hardware part of the lower computer of the system,different sensor selection and circuit design are carried out according to the required information acquisition channels.At the same time,in order to meet the needs of miniaturization and portability of data gloves,the relatively complex multi-channel acquisition circuit is optimized,and a multi-channel traversal circuit is designed to reduce the volume of the circuit.The circuit PCB and the glove hardware shell were drawn,and the lower computer system was integrated on the glove base to make the data glove.In the software part,the operation program of the lower computer is designed to meet the signal acquisition of different sensors,and the collected data is optimized by weighted recursive average filtering,normalization algorithm,Kalman filtering and other filtering algorithms,and the appropriate data format is designed to meet the real-time communication between the lower computer and the upper computer.The upper computer software part uses MATLAB to present the transmitted data in real time in the form of graph combination for hand data monitoring,and uses Unity 3D for virtual scene modeling to realize the user’s hand action behavior repetition in the form of virtual hand.For the purpose of motion authenticity,the mathematical model of hand knuckles was established to realize multi-joint Angle mapping,and static and dynamic gesture libraries were established to satisfy the basic gesture recognition work.Through the experimental analysis of the designed multi-mode sensing interactive data glove system,the accuracy of signal acquisition of the glove system,the authenticity of data presented by the upper computer to the virtual scene,the effect of basic gesture recognition is good,and the system is stable.The multi-mode sensing interactive data glove system designed in this paper has abundant sensing channels,low production cost,flexible use,wide application scenarios,and scalability.At the end of this paper,the shortcomings of the data glove and the direction of further optimization and improvement are also described.