| Grasping and controlling objects is an important function of the hand.In order to achieve accurate and stable grip,human body needs to receive environmental information in real time through various sensory systems such as vision and touch,and form decision through the integration of central nervous system,then send out movement command to peripheral nerve,and complete the grasping action by stimulating muscle contraction.In the process of pricision grasping,it is of great significance for the study of neurophysiology to research how the brain integrates the tactile and visual information that is essential to this behavior and realizes the closed-loop control of object grip control.However,the traditional research methods can only simply overlap the effects of touch and vision.However,due to the limitation of technical means,visual and tactile information can not be separated,and there is no conflict between them.A key problem that can not be explained is,when the visual and tactile information is inconsistent,will the grasp control be more sensitive to the perceptual information of a certain mode?and whether the role of vision and tactile in the control of grasping is the same,etc.The research on these problems will help us to understand how different modes of perception information such as vision and tactile play a role in precise motion control,which is important to clarify the working mechanism of human nervous system,to establish noninvasive detection method for complex neuromuscular diseases,and to a new generation of bionic prosthetic robots with two-way perceptive motion pathway.In order to solve this problem,this paper focuses on the development of a grasp test and analysis system based on virtual reality.It can simultaneously record the precise finger force,multi-channel EMG and EEG signals of each finger,and produce quantitative visual tactile conflict,objectively and accurately record the signal of behavioral and neurobiology.At the same time,the paper studies whether the visual tactile conflict can be enhanced by multi-channel vibration feedback to compensate for the impact of visual tactile conflict.Based on the above technology development,the research further designs the multi finger precise grip experiment under the mixed reality environment.By grasping the cup model,the daily behavior and the difference of tactile and visual tilt angle of the water cup are observed,and how to integrate the conflict information of different modes of vision and touch in the central nervous system to maintain the relatively stable mechanism of the water cup.Specifically,the work of this study includes the following aspects:(1)Platform development.In this study,the virtual reality device HTC VIVE is used to develop the interactive environment of virtual reality.The virtual cup model is added additional rotation in X and Y directions respectively to generate the interaction mode with inconsistent vision and tactile feedback.The tracker attached to the water cup provides the real-time position and posture of the water cup,and the real-time feedback of these information also forms the synchronization of virtual and reality.The system integrates the interfaces of EEG,EMG,force and torque data at the same time,and implements simultaneous decoding and synchronous storage in various ways.In order to make the experimental platform run stably and robustly in human experiments,many classic design patterns are used to ensure the independent and safe transmission and sharing of physiological signal data of various types and parameters.The software system uses multithread concurrent pattern to keep communication and synchronization with other data interfaces at the same time,and different types of data use corresponding coding and decoding algorithms to maintain the balance of speed and effect.The hardware system uses MCU as the lower computer to communicate with the upper computer and decode the command of the upper computer,and then drive the vibration motor to provide vibration feedback.The communication protocol group and filter driver circuit designed by ourselves can meet the needs of the experimental environment.The encoding and decoding library VrParser,which is designed with pure C++code,provides a cross platform interface for the encoding and decoding of binary data in the experiment process.Users of the platform can expand and integrate this code library in a very simple way,such as connecting it to the runtime library of Matlab and Python.(2)Behavioral data analysis.In the analysis of behavioral data,this paper analyzes that different information weighting strategies lead to completely different results under the condition of vibration feedback and no vibration feedback.Vibration feedback can effectively reduce the tilt angle of the cup(3.980 vs 1.71°,p<0.01),and the extra tilt angle of the cup also significantly affects the tilt of the cup,which is in line with the expected fact.Then,the possibility of mixed Gaussian modeling for tilt angle is proposed.The title angle data can be divided into two types of Gaussian distribution with or without vibration feedback.We think that this is the result of two types of differentiation of grasping strategies.However,the boundary between the two types of Gaussian distribution is not clear,which also reflects that the sensory allocation strategy for grasping is not discrete,but continuous.In the analysis of thumb force in the experiment,the fitting area of the pressure center point has no significant change(p>0.05),but the entropy of finger force decreases in y direction when vibration feedback was provided(p<0.05),which may be caused by the vibration feedback added to the opposite limb also occupied a certain proportion of tactile feedback channels.(3)Electroencephalogram and Myoelectricity data analysis.In the calculation of EEG power change,we found that different conditions have significant effects on the brain activity in beta band.Specifically,the addition of vibration feedback leads to lower beta activity power(-15.16,p<0.01),and the inhibition effect of visual cortex is much higher than that of bilateral sensory cortex(-35.48,p<0.01),but there is no significant difference between the two sensory cortex(-7.67,p=0.40).By comparing the differences of brain activity between the two Gaussian distributions mentioned above,we found that the experiment with higher mean Gaussian distribution had a higher degree of beta activity inhibition(-39.84,p<0.01).For the higher frequency of gamma activity,no difference of experimental factors was observed.In addition,we also observed the different coherence degree between the gamma band near C3 and the beta band near C4 and the muscle of the upper limb,and both of them were higher under the condition of no vibration feedback(t=2.56,p<0.05).In the EMG analysis,we found that the vibration feedback caused the EMG amplitude and median frequency to move to higher values. |
PDF Full Text Request