The Design And Implementation Of Video Mixer Based On Image Code Stream

Video Mixer, which provides multiple-video-windows composing service for conference participants, is a crucial part in video conference system. For long, Video Mixers are used to be implemented by specific hardware. But high prices of hardware hinder the popularization of video conference system. Therefore, to implement video conference system in pure soft solution could be the future. According to bandwidth and server resource limitation in low-end application circumstance, this paper presents a design and implementation of a soft video mixer, and so as its performance tests.Basing on requirement analysis in resource limited application circumstance, this paper proposes the framework of an image code stream based soft video mixer, which is mainly consisted with mixer, frame synchronizer, transmission control module three parts. The mixer catches image properties and image entropy coding byte stream through extracting attribute bytes from image code stream, and then completes multi-video combination by splicing image tiles. That avoids server's image coding-decoding operations, so as to release computing burdens of the server, in which a hash table in maintained to record video requirements from different users. Within the frame synchronizer, it synchronizes video images of different senders through computing timestamp information in received RPT frames. The transmission control module uses redundancy encoding to protect video stream from lost package. When packages lost, it uses the redundancy information of next package to recover the previous one.In implementation part, this paper first describes the image pretreatment process, gives the method of using JPEG2000 code stream truncate operation to modify input image resolution, and how to acquire structures of code stream class through image code stream feature extraction. Then the image tiles splicing part presents the process of splicing image tiles through canvas size computing, image tiles index computing and image payload composing. At last, it describes the process of sending and receiving video data through RTP.Basing on Java Media Framework, this paper implements a code stream based video mixer, and tests its performance on a resource limited platform. Test results manifests that, compare to traditional processing method, a code stream based video mixer has a lower logical CPU consuming value when processing video composing task. That proves a better effect.