Study On Synchronization Method Of Video Embedded By Audio In AVS

With the development and popularization of Internet, the communication manner of people has been developing from traditional voice to multimedia communication including voice, text, image and video. With the diversification of communication modes and the sharp development of the application in every field through the Internet, the quantity of multimedia information in Internet has been growing rapidly, there into, the multimedia information of video and audio is increasing the fastest. Thus, the real time synchronization of video and audio in multimedia communication has practical meaning. The main problems to be solved in real time synchronization transmission of video and audio are synchronized collecting, compression, synchronized transmission and information receiving, and synchronized play back of video and audio. The final intention is to realize the synchronized play of image and audio in the both client end and collection locale. The so-called synchronization of video and audio is that the time delay of image we saw and the audio we heard is little. The real time is to decrease the delay of the play end and the collection end, so people will not have obvious perceive. In addition, in applications of multimedia database structure, retrieval, searching and browsing, the synchronization of video and audio is also needed to realize the synchronized interactive retrieval and query of video and audio.In the series of MPEG standards, the coding of video and audio are performed independently. The synchronization of bit streams between video and audio is achieved by using timestamp. The decoder can recover a local system clock coherent with encoder according to program reference clock by using a lock loop. In the series of MPEG standards, the display timestamp is based on a perfect decoder. The perfect decoder is supposed that the channel delay can never overflow or underflow and the processing of bit stream is instantaneous and ideal. Therefore, if the program reference clock and display timestamp can be encoded in the encoder correctly, stored and transferred without error and decoded in the decoder accurately, and the decoder can recover the system clock coherent with encoder based on program reference clock and display image and audio in time of correct timestamp, then the synchronization of video and audio can be achieved. However, the above conditions are inappeasable for a practical decoder. Therefore, various improved synchronization schemes of MPEG have been proposed, such as audio and video synchronization scheme based on real-time transport protocol (RTP) by making use of dynamic reordering, decoder recovering, and adaptive synchronization mechanism, synchronization scheme by controlling buffer configuration and buffer size, synchronization scheme which is achieved by adjusting video display frame rate based on audio display time, and synchronization schemes based on synchronization generation and adaptive interleaving strategy. All above schemes need the exact timestamp. But in the network environment of Internet, the IP protocol can only provide a"best effort"service, which means the transmitted data can have different arrival delay and even be lost. Accordingly the program reference clock and the display timestamp may be error in transmission. Thus these synchronization schemes of video and audio are not suitable for the transmission, storage and playing of multimedia communication in Internet.Based on widely referring to the materials about video and audio coding standards of AVS and the synchronization technology of video and audio, this thesis discussed the status and development trend of video and audio synchronization in home and abroad at present and investigated the major techniques and analyzed the major synchronization method which is designed by embedding audio data into video in detail. Also, the thesis investigated the embedding overhead brought by implementing video embedded by audio in AVS in detail. In sum, our works are organized as follows.Firstly, we describe the technology of synchronization, analyze the necessity of synchronization between video and audio, and formulate our further research direction by considering the development of AVS and the synchronization method in MPEG-2.Secondly, the research work of the method of video embedded by audio in AVS is explored very widely, and the technology of embedding in the encoder and the extracting in the decoder is investigated deeply. Studying on synchronization method of video embedded by audio in AVS is to research how to embed the AVS compressed audio data into the AVS video coding system to achieve"always synchronization"of video and audio in transmission or storage, decoding in the receiver and the process of playing and how to perform related extracting, as well as provide the whole system synchronization scheme. The key point is to design an embedding arithmetic, which must obey the audio preferential rule, that is, the embedding and extracting of audio bit must have no error. Meanwhile the influence to the video must be very little. From objective aspect, the decrease of PSNR must be less than 0.5dB. From subjective aspect, the images do not have obvious difference. Furthermore there is an important design target that the embedding overhead can not be substantive when implementing embedding. The channel overhead in MPEG-2 is 3%. So the embedding overhead of the proposed method must be less than 3% in order to become a practical technology.According to all the demands above, this thesis designed three fixed point embedding methods and one adaptive embedding method. By analyzing experimental results of the proposed embedding algorithms, it can be concluded that the audio data can be embedded and extracted without distortion. The influence of embedding to the video is little enough to be ignored. From objective aspect, the decrease of PSNR is not larger than 0.5dB. For the best algorithm from the 4 proposed embedding methods, the PSNR decreases 0.04dB in minimum and 0.17dB in maximum. From subjective aspect, it is hard to perceive difference between the reconstructed images with audio embedding and without audio embedding, that is, human eyes can not perceive the influence caused by embedding From the aspect of embedding overhead, the best case of the proposed 4 methods has embedding overhead of 0.35% in minimum and 1.64% in maximum. Comparing with the 3% channel overhead needed by synchronization using timestamp in MPEG-2, our method can save much system channel overhead.Finally, the effect and performance of the proposed synchronization method of video embedded by audio in AVS is summarized and the further work orientation is pointed out.