Research On Haptic Interface And Interaction Technology For Image Display On Touch Screen

Touch screen-based smart devices have been widely used in people's daily lives.The touch screen has the function of displaying visual data and detecting the touch position of the finger,which provides a new method for people to interact with the virtual environment.Typically,a user interacts with the touch screen device through sight and hearing,but the haptic interaction is very limited.Haptic interaction is a new type of human-computer interaction technology with a bidirectional information transmission capability.It enables the operator to touch,feel and manipulate virtual objects,and simultaneously display various haptic-related features information such as surface friction,texture,softness,and material of virtual objects to the operator.Especially for blind or visually impaired people,haptic interaction provides an important way for them to perceive a variety of digital information such as graphics,images and virtual environments displayed on the touch screen.However,current human-computer interaction technologies for touch screen applications generally lack effective haptic sensations,and seldom consider the haptic expression methods that conform to human physiological perception characteristics.In view of the characteristics and actual needs of touch screen interaction,this paper firstly combines the haptic interaction technology with the touch screen device from the physiological sensing characteristics of human beings.Three haptic devices,including hand-held and finger wearable,are designed separately,which can display various feature information of a virtual object or image through multi-mode haptic feedback.Then,in order to better help people perceive the spatial information of the image on the touch screen,this paper first applies the image smoothing algorithm based on deep learning to smart devices to smooth the original image before haptic display.Finally,the performance and characteristics of the three haptic devices are tested by the haptic display experiments of the contour and shape of the image,and the effectiveness of the proposed haptic expression methods are verified.Specifically,for the shortcomings of lack of sufficient force feedback and single display mode for the touch screen interaction,a pen-type haptic interface device MH-Pen with multimode haptic feedback function is designed.The device integrates four types of actuators internally,which can simultaneously transmit force information and tactile information generated in the interaction to the user.By combining a small passive force feedback actuator based on magneto-rheological(MR)fluid with an active actuator,the device provides a wide range of precise feedback forces for interaction.In the display experiment of virtual stiffness,improving the precision of force feedback and using multi-mode haptic feedback significantly improved the stiffness display performance of the device and the subjective perceptual resolution of the subject to virtual stiffness.In order to provide local variable lateral force feedback for the user to slide on the touch screen,a passive spherical actuator is designed by utilizing the electrically conductive and magnetically conductive properties of the magneto-rheological elastomer(MRE),and its high coefficient of friction with the touch screen.In the implementation of the actuator,an isotropic natural rubber-based MRE is first produced.Next,based on the magnetic permeability and surface friction characteristics of the MRE obtained by tests,a lateral output force model of the actuator is established.Then,the finite element analysis is used to obtain specific structural parameters,and the prototype is manufactured.Finally,the current control method for determining the lateral output force of the prototype is determined by a calibration test.Thanks to its ability to move with multiple degrees of freedom and the ability to interact directly with the touch screen,the actuator is well suited to the needs of multi-person and multi-point interactions.Inspired by the wearable haptic system,a finger wearable haptic interface device FWTouch is developed for touch screen interaction.The device has normal force,lateral force and vibrotactile feedback capability,allowing the user to comprehensively perceive information such as friction,hardness and roughness of the surface of the virtual object through active exploration such as sliding and pressing on the touch screen.In addition,in order to generate sufficient normal force feedback,a small MR foam actuator without seal is designed,and the force control strategy based on Hall Effect sensor is used to correct the output force of the actuator.In order to better display the spatial information of the image,this paper uses the image smoothing method based on convolutional neural network to obtain the main structural features of the image.The method is capable of simulating the image smoothing effect of the relative total variation filter and significantly reducing the time taken for image smoothing.The shape and contour features of the smoothed image are then extracted and the spatial features of the image are converted into a form suitable for expression by force or/and tactile.On this basis,experimental studies on haptic display of the spatial features of the image are performed on the touch screen using the haptic devices developed in this paper.Firstly,by transforming the MHPen,this paper proposes a contour display method based on motion direction guidance,which significantly improves the accuracy of contour recognition of subjects and reduces the time consumed by the perception process.Then,through contrast experiments,this paper compares the shape display effects of the three devices,and for the first time studies the performance of the lateral force-based tactile illusion theory to display image shape features on the touch screen.The experimental results show that the lateral force field has a good shape display effect only for images whose waveform gradient continuously changes,while the displacement field can obtain better shape display effect for different types of images.In addition,the results of this study also show that image smoothing plays a very important role in improving people's ability and accuracy to sense and understand image spatial features.