Research On Audio-video Synchronization Coding Based On Inter Prediction Blocks Of Varying Size For H.264

In recent years, the progress of computer technology and communication technology promotes the fast transmission of digital information product. The information obtained changes from individual form to multimedia, and produce a new communication type—multimedia communication. Such communication technology appear in everywhere, and more and more multimedia service will be a part of our lifes, such as video conference, video broadcast, network television, long-distance teaching, multimedia net game, Computer Supported Collaborative Work (CSCW) and so on. A key point of all there applications is controling the synchronization between audio signal and video signal in multimedia communication. So the study on synchronization between audio signal and video signal has practical meaning. Usually a multimedia system includes five parts:data colletion, compression coding, network transmission, decoding, synchronous play. Multimedia synchronization technology mainly focus on the synchrous controling beween audio signal and video signal in the procedure of data collection, transmission, receiving. And the final goal is obtaining the synchrous play between the audio signal and video signal at the receiver. For solving this problem, a synchronization measure is proposed.The classical algorithm for multimedia synchronization is timestamp synchronization method and synchronization mark method. Timestamp synchronization is interting uniform timestamp into streaming media data unit to get time synchronization. And the correlated media streams have the same time stamps. At the receiver, there correlated media streams will be play at the same time, realizing the synchronization btween the different media streams. In timestamp synchronization method, audio stream and video stream are transfer individually, so a system clock is need to control synchronization at the receiver. But with the jitter in the network, It is hard to make sure the accurate system time, which is main disadvantage of timestamp synchronization method. And this method can not assure audio-video synchronization at the transmitter.This paper proposes a multiplexing synchronization method,for solving the complete audio-video synchronization at the transmitter,channel and receiver. In this method, audio stream is multiplexed into video stream,so the time relation between audio unit and video unit is maintained. The complexity for controling synchronization at the receiver is reduced. This method always considers the multimedia stream's characteristics and syntax for not destroying the data. So this kind of methods is based on a particular video coding standard.The synchronization method proposed in this paper is using H.264 video compression coding standard to insert the audio stream into the video stream. As the most advanced video coding standard, H.264 have many advantages such as good coding flexibility, network friendliness and high compression effciency. H.264 video compression coding standard is the draft developed by the Joint Video Team(JVT), which can provide more acceptable visual quality than earlier standards even at low bit rates. H.264 video syntax is broken into two layers—video coding layer(VCL) and network abstraction layer(NAL). VCL consists of the bits associated with the slice layer or below– the primary domain of the compression tools, including two parts: encoder and decoder for compression and decompression. Accordring to coding function and application area, three profiles are defined, which are the Baseline,Main,and Extended Profile. The Baseline profile was designed for low delay applications with low processing power and in real-time environment, so we introduce Baseline profile to our synchronization system. In Baseline profile, H.264 supports many features as follow: Multiple reference picture, deblocking loop filter, inter-prediction blocks of varying size motion compensation, integer transform. The synchronizaiton method in this paper using the variability size of matching blocks in block mode selection process, make the selection depend on the audio streams rather than on the coding cost,achieving the synchronization. And at the receiver, the synchronous play effect can be obtained through the decompression of video stream and audio stream.Besides, the interior data hiding method is applied into multiplexing synchronization, generating a new embedded audio-video synchronization coding method, for synchronized compression of audio and video. This method combines the embeding process and H.264 video compression coding standard, using audio signal as the embedded information insert into the H.264 the inter prediction process. The advantages of this method is the visual undetectability and little affection of compression efficiency. Furthermore, the embedded synchronization method make the audio-video synchronization goal achieved at the transmitter terminal.On the basis of the overall understanding of H.264 video compression coding standard, I focus on motion vector prediction, blocks mode selection in the inter-prediction, proposes a audio-video synchronization coding based on inter prediction blocks of varying size for H.264. In this method, the embedded position is those macroblocks in the P frame choosing mode P1,P2,P3,P4 as the best prediction mode. And every macroblock can embeds two bit audio singal. The embedding scheme uses the mapping rule between audio signal and matching block in the inter prediction to embed audio: after being converted into binary, every two bits of audio signal map on to a partition motion partition mode and a prediction mode. Then through modulating the prediction mode in the inter prediction, the synchronization can be achieved.In order to prove the feasibility of our algorithm,this paper uses H.264 codec-JM to realize concrete encoding/decoding. JM is an official standard codec,and it is the most widely used open source codec. The algorithm is tests six different standard sequences based on JM software,and using the subjective quality,PSNR,embedding overhead as the standards of analysis. The experimental results show that after using this synchronization algorithm, mbedded systems brought little overhead,cannot affect the synchronizaiton transmission and saving.. The impact of video is small, changes in the subjective test is also very difficult to detect and the PSNR drop can be ignore. And this a algorithm have no impact of audio.