Phonemic Restoration Based On The Movement Continuity Of Articulation

Phonemic restoration effect is a perceptual phenomenon where under certain conditions,sounds actually missing from a speech signal can be restored by the brain and may appear to be heard.This illusory phenomenon reveals the sophisticated capability of the brain underlying robust speech perception in noisy situations.There are two accepted two hypotheses: the first hypothesis is coarticulation;and the other one is semantic context provided by sentences.These two hypotheses fight with each other and neither of them has been verified effectively.For the first hypothesis,when the human being producing speech signal,the articulation organs such as lips,a tongue,and a jaw,cannot move freely and abruptly,instead,these organs move smoothly in a highly cooperative manner,resulting in the articulation of adjacent phonetic segments interacting with each other.And the human being brain perceives the continuity on articulation to realize phonemic restoration.An effective method based on this consideration to retrieve missing speech signal and makes the relevant hypothesis verified to some degree.The proposed method is as follows:First,endpoint detection is conducted on the speech signal in which some parts are lost or damaged.Then,acoustic feature extracted from the existing parts is mapped to the corresponding articulatory feature.After arraying these articulatory feature parts in the order of time sequence,we can utilize the movement continuity of articulation to do interpolation on them to get the missing articulatory feature parts.At last,mapping the obtained articulatory feature parts to the corresponding acoustic feature and synthesizing speech to fill in,we can retrieve the missing speech signal.To realize the proposed method,a bidirectional mapping model is established based on deep neural network(DNN),in which,different articulatory features are studied for the mapping model and an improved architecture hierarchically concatenating bottleneck feature and articulatory inversion component DNNs are proposed;and then missing articulatory feature obtained from missing speech signal is restored with cubic spline function.At last,25 sentences are selected from the database MNGU0 and short durations of the sentences are replaced by zeros and/or noise for evaluating.Experimental results show that the proposed method can effectively improve perceptual evaluation of speech quality(PESQ)of the speech with missing signal.And these experimental results provide preliminary experimental clues for verifying the first hypothesis of phonemic restoration—coarticulation.