Use Of A Deep Autoencoder To Create New Features From Raw Images For An Ultrasound Based Silent Speech Interface

Silent speech communication is a way to track the movement of vocal tract to understand the spoken words without audible sound,and the most common application is lip reading.For those who has acquired Articulation Disorders,silent speech recognition and synthesis system can effectively solve the problem of their communication.On the condition of that,Silent Speech Interfaces(SSI),based on the non-acoustic capture of un-vocalized speech,promise to enable secure,reliable voice communication in quiet public places as well as in noisy environments.Solutions using a variety of sensor have appeared in the literature,including ultrasound(US)images of the tongue and video images of lips;electromyographic electrodes(EMG)attached to the facial area;and electromagnetic articulography(EMA)sensors attached to articulators.In this paper,we build a Silent Speech Recognition system based on ultrasonic imaging and video images of SSI,and implement the transformation from silent speech signals to the text results.This paper has first proposed to apply DNN to SSI,which is based on ultrasonic imaging,recognition results shown a great improvement compared to the benchmark system.Silent Speech Recognition system is mainly divided into two parts: the non-acoustic feature extraction and speech recognition.In non-acoustic feature extraction,we proposed to use Autoencoder,rather than linear transformation method,and reconstructed image results are much better than those of DCT;then using extracted features as the input of DNN-HMM model training,the recognition results improved over the benchmark.Besides,the recognition results and performance of information compression from DAE features are better than DCT's.At present,the DAE features have joined the Silent Speech Challenge database,became a new non-acoustic features in ultrasonic imaging based system.With the rapid development of mobile computing,SSI has much more applications in the future,such as secret communication in mobile devices.In addition,it can provide people the health information about their vocal tract through movement.