Ultrasound Image Analysis For Silent Speech Interface

In recent years,the design of a device allowing people to communicate without speech signal has been considered of great significance.The portable device is called Silent Speech Interface(SSI).The implementation principle of the SSI is to acquire the motion data of the voice organ through the sensor during the production of silent speech,and then the data can be processed and modeled to achieve the conversion of voice signal.The SSI is used as an alternative to tracheo-oesophageal speech for larynx cancer patients,for preventing the disclosure of military,for safeguarding personal privacy in public,for communicating under the condition where silence must be maintained,for communicating in the noisy environment,or for reducing the encoding rate.Ultrasound images of the tongue contain more acoustic information than the optical images of lips,thus the feature extraction of ultrasound images is a crucial part of SSI.Raw images cost too much calculation to build the SSI,so it is necessary to develop a feature extraction method which preserves critical information.This paper presents three hybrid feature extraction algorithms.The first hybrid algorithm is called HWT-PCA,which performs Principal Component Analysis(PCA)to reduce Haar Wavelet Transform(HWT)coefficients dimensions.The second and third hybrid methods are called block DCT-PCA and block WT-PCA,separately,in which the DCT or WT coefficients are truncated to the appropriate number according to energy,and use PCA to reduce the dimensions of the selected coefficients.And then this paper proposes a feature fusion method which use Local Binary Pattern(LBP)and DCT to extract the feature of ultrasound images.The feature fusion method can combine the information of the frequency domain and spatial domain to acquire better feature.Visual observations of each phonetic class are modeled by continuous Hidden Markov Models(HMMs)after feature extraction.The experiments are based on an audiovisual database containing 90 minutes of continuous Chinese speech from one speaker.The experimental results show that the proposed hybrid feature extraction algorithms and feature fusion algorithm can extract the feature of tongue ultrasound image effectively.These algorithms can be applied to the SSI to increase the usability of SSI.