Research And Implementation Of Audio And Video Synchronization

With the developments in communication, broadcasting, TV and computer technologies, it is feasible to provide multimedia services such as Video Conference Systems, Remote Teaching System and Viewed telephone, by which the people that live in different area can intercommunion as front to front. Compared with traditional data applications, multimedia services are characterized by the integration of different types of media with at least one time-dependent medium. There are some temporal relationships among the information units of these media streams, which must be maintained during their storage, delivery and presentation.Other than circuit switching, the Internet provides a best-effort sevices, i.e.the data can come across unpredictable delay, or lose. Jitter and packet-loss destroys temporal relationships between media units which constitute a real-time media stream, thus hindering the comprehension of the stream and reducing internet real-time multimedia application effect. An important objective of the research community has been to devise methods that cope with the variations of the network delay-also called jitter that are an inherent characteristic of best-effort networks. Obviously, multimedia synchronization is a special and important quality of service (QoS) in multimedia systems. As a key technology in multimedia communications, research on multimedia synchronization has received a wide range of attention in the recent years.Multimedia synchronization consists of intra-media synchronization and inter-media synchronization. At present, Research about multimedia synchroniza- tion is also a new domain; it is mainly done in two aspects: description and realization of media synchronization. Back with in recent years, research in description makes progress, such as time-axis sync, virtual-axis sync, hierarchical sync based control flow, Reference point sync, event sync and script sync. Then the research of implementation of media sync is concentrate on opened and distributed heterogeneous environment.Multimedia synchronization refers to the data's producer, transmission, and play. It can be controlled in transmitter, network or receiver, or controlled in allthree ways simultaneously. This paper mainly deals with continuous media synchronization, especially the sync between audio and video in transmitter and receiver, this is the most base and important technology.The mainly research work and results of this paper is listed as follows.1. In the paper, the history, developed and related concepts of multimedia synchronization are fully reviewed. We mainly introduce the four layer "waterfall" type synchronization reference model set by Blakowskia and Steinmetz, generalize many synchronous mechanisms put forward in recent years, finally introduce correlative multimedia protocols and synchronization measure standard.2.This paper designs an audio and video real-time collection and direct- playing system, which includes four modules: equipment interface module, code-decode module, network interface module, synchronization control module, then describes particularly functions of every module and arithmetic of media data code-decode. This system collects audio and video data in real time, transmitted by UDP means which is unreliable and oriented connectionless. we can say that it is rudiment of many today's multimedia application such as multimedia conference, audio and video supervise system, it is a primary platform of which we make use to do research about multimedia synchronization.3. This paper does some research for intra-media and inter-media synchroni- zation for continuous media streams. To intra-stream synchronization, mainly discuss the affect which presentation schedule may do to it. The presentation schedule used to realize synchronization can be generally classified as two categories: time-oriented schedulers and buffer-oriented schedulers. The time-oriented schedulers make use of time-stamp of MU (media unit) and clock of both transmitter and receiver to measure network delay and jitter; the buffer-oriented schedulers make use of level of buffer engrossed to indirectly evaluate jitter of current network To inter-syncronization, mainly analyze some delay models such as FIFO (first in first out) model and audio-first module, and a deduction and comparison about them is given. Also some other researchers'synchronous mechanism is analyzed and summarize.4. According to the research result, the thesis provides an audio-video synchronization solution scheme which is based on the real-time transmissionprotocol RTP/RTCP. The scheme makes use of RTP/RTCP and the time-stamp mechanism to realize the audio-video synchronized control. RTP/RTCP is one of the core protocols in the network multimedia application. When ITU-T develops the multimedia communication standard H.323 on the packet network, it is the key protocol to guarantee the QoS of the multimedia stream. Some mechanisms such as time-stamp, serial number and load identifier can be used to synchronization control. The thesis uses the open-source library JPTPLIB to implement the RTP, design and realize the RTP packets of audio-video stream.The scheme adds the RTP mechanism to the system by adding the control message such as time stamps to the audio and video transmission packets separately. Based on the intra-media synchronization, we choose the audio stream as master stream and the video stream as slave stream. According the corresponding relationship between the time stamps of audio and video data frames, we determine the synchronization points of different packets. Then we adjust the procedure of the video playing to realize the effect of media synchronization. The experiment results show that the scheme we propose can efficiently realize the synchronized communication between audio and video.Generally, we do some research and deduction in the thesis, which mainly talks about the multimedia, especially the synchronization mechanisms of the voice and video data stream. Based on the understanding of this mechanism, we design a scheme to implement the audio-video synchronization, and do some tests on the PC platform. Based on the test results, we can conclude that the scheme has achieved fine effect. However, there are still some problems need to be perfected. The problems such as group synchronization of multi-media, combined synchronization of storage media and real-time media, synchronization when there occurs user-interacting, multi-media synchronization on the embedded system still exist. They need to be further studied.