Research On Acoustic Modelling And Text Generation In Concept-to-Speech Conversion

Speech synthesis aims at enabling the computer to speak and mainly includes Text-to-Speech (TTS) and Concept-to-Speech (CTS). Among the different modes of speech synthesis, TTS allows the computer to read text aloud. However, more than just reading text, human beings can also compose and articulate sentences based on ideas in minds. Translating this ability to computer relies on CTS.CTS converts conceptual representation of the sentence-to-be-spoken into spo-ken utterance. While some CTS systems consist of independently built text generation and TTS modules, most of the existing CTS systems enhance the connection between those two modules by incorporating a prosodic prediction module that utilizes linguistic knowledge from the text generator to predict prosodic features for TTS. However, the CTS system can be further improved if more knowledge embodied within individual modules can be shared in more effective ways.This paper explores methodologies to optimize the acoustic modelling and text generation modules in CTS based on sharing information. The firs method utilizes the linguistic knowledge offered by text generation module to improve the Hidden Markov Model (HMM)-based acoustic modelling in the CTS. The second method further elimi-nates the original prosodic features in the model context of HMM-based acoustic model in order to avoid prosodic modelling in CTS. This method circumvents the prohibitive cost for constructing prosodic model and possible erroneous prediction from prosodic feature prediction. The last method optimizes the text-generation method based on eval-uating the quality of synthetic utterances corresponding to multiple candidate sentences that can express the input concept and selecting a superior candidate as the sentence-to-be-synthesized.This paper includes the following chapters:Chapter1will introduce CTS and compare it with TTS after short discussion on human’s potential on language. Then this chapter will also summarise the research on CTS.Chapter2will explain the theory and implementation for all components in the baseline Mandarin CTS system, including natural language generation, external prosod-ic modelling for Mandarin and HMM-based parametric speech synthesis. After that, this chapter will discuss the disadvantages of the baseline CTS.Chapter3will discuss the first method to improve the HMM-based acoustic mod-elling in CTS through incorporating syntagmatic information offered by the natural lan-guage generation module. Experiments show that the accuracy of the predicted FO tra- jectory can be improved by incorporating syntagmatic features.Chapter4will show the second approach that replaces the symbolic prosodic fea-tures with the syntagmatic features in the model context of the acoustic model. Experi-mental results show that this approach can avoid the laborious prosodic modelling while achieving comparable performance to the traditional approach.Chapter5will explain the text generation method. At first, multiple candidate sen-tences that can express the input concept will be composed. Then K-Nearest Neighbour (K-NN) will be utilized to evaluate the quality of syllables in all the synthetic utterances corresponding to the candidate sentences. After that, rule-based algorithm will evalu-ate the quality of the synthetic utterances. Based on the evaluation results, superior sentences can be identified as the final candidate to be synthesized and delivered as output. Experiments will show that the CTS can deliver better synthetic utterance to express the input concept.Chapter6will draw the conclusion and list several aspects that deserving further investigation on CTS research.