| Tactile feedback technology can greatly enrich the reality of human-computer interaction and improve the efficiency of human-computer interaction.Research shows that the reality of tactile feedback is a key factor to determine the user experience,and the organic integration of tactile feedback,hearing and vision has important application value in multimedia terminals.In the multimedia interactive environment,tactile feedback technology is introduced,so that the operator can not only get audio-visual experience,but also experience more realistic interactive process such as shooting,collision,game skill release through tactile feedback,and broaden the perception dimension.With the development of human-computer interaction,multimedia terminals with mature audio-visual combination technology have been widely used in many fields.On this basis,the combination of hearing and touch has also attracted extensive attention and exploration of research teams at home and abroad.It has a broad prospect to study the relationship between audio signal and tactile signal,and to render audio signal by force and tactile.The realization of audio signal haptic rendering needs to solve two problems,one is the universality of the extracted haptic features,to ensure that the feature extraction method can be applied to more audio signals,which is a necessary condition for audio signal processing,and the difficulty lies in the selection and extraction of various features of audio signal;the other is the establishment of mapping relationship between haptic features and haptic effect This is the key of tactile rendering reality.The difficulty lies in the establishment of the mapping relationship between hearing and touch and the guarantee of tactile feedback reality.There are three kinds of audio signal force tactile feedback devices,which are wearable,mobile terminal and seat type.This paper focuses on the haptic feature extraction of audio signal and the relationship between audio signal feature and motor loaded haptic driving signal based on mobile terminal(1)A haptic rendering method of audio signal based on feature extraction and mapping is proposed.According to the relationship between human hearing and tactile perception,this method maps between hearing and tactile perception,so that the tactile feedback effect is better and closer to the real auditory perception.Taking the frequency and sound pressure level of audio signal as haptic features,aiming at the relationship between audio signal,motor driving signal and human perception intensity,two mapping relationships of audio frequency sound pressure level and driving frequency,audio frequency sound pressure level and driving amplitude are established respectively.According to the mapping relationship,the haptic rendering method of audio signal is obtained,and the conversion from audio signal to haptic driving signal is realized.(2)A portable,compatible and low delay tactile feedback system is built.The hardware part of the system is composed of mobile phone terminal,sound card,driving circuit and linear vibration motor.The design based on mobile phone terminal makes the system more portable;the software part is based on C + + programming language,which can be compatible in all kinds of software development environment.The system can read the audio signal from the mobile terminal in real time,extract the tactile characteristics of the audio signal,calculate the tactile feedback signal,synthesize the vibration tactile feedback according to the rendering method,and control the system delay within 50 ms,which provides a platform for the implementation of the tactile rendering method proposed in this paper.(3)It is verified that the tactile rendering effect of the proposed method is realistic.By comparing the proposed method and a machine learning based audio tactile rendering method,the tactile feedback effect of pure tone audio signal rendered by the two methods and the tactile feedback effect of 33 kinds of gunshot signals in real game scenes,it is verified that the rendering method proposed in this paper can better reproduce the perception of audio signal at the tactile level,and obtain a higher sense of reality.User experiments include subjective evaluation of audio signal tactile rendering effect in pure tone audio signal and real-time game signal.By analyzing the advantages and limitations of the method proposed in this paper,the conditions for its use are explained. |
