Embedded Multimedia Communication Terminal Based On Dual-MIPS Processor Platform

With the rapid development of the computer science, network communication, video and audio processing technology, multimedia communication system is applied abroad in many fields, such as video phone system, video on demand system, video conferencing system, video surveillance system, tele-education and telemedicine system. So, multimedia communication terminal,with full duplex multimedia encoding, decoding, and communication capability, has become to be a development tendency of current information society.An embedded system consists of an embedded processor and an embedded operating system. Embedded operating system runs on embedded processor, it can dispatch all tasks and manage the processor and other hardware devices of bottom layer. As a general-purpose embedded operating system, Linux is powerful enough, but it sacrifices its realtime performance. uC/OS-II is a realtime embedded operating system, but it is too weak.VIVO Micro's VM600 high-performance multimedia processor is designed with a cutting-edge architecture–"Dual-CPU"especially fitting the requirements of multimedia communication terminal. It integrates powerful performance with two MIPS 4Kec RISC cores, which delivers high data thoughput, realtime video and audio processing, and data routing functions. One MIPS core processes realtime multimedia streams under uC/OS-II, and the other core deals with network communication and terminal application under Linux operating system. According to these features, embedded multimedia communication terminal is studied in this paper. An embedded multimedia communication terminal based on dual-MIPS processor, uC/OS-II and Linux operating system is designed and implemented finally.The terminal is designed based on VM600 SoC processor, uC/OS-II 2.52 and Linux 2.6.19 operating system. It adopts SIP/SDP protocol to initialize sessions with other terminals and describe the media format, and adopts RTP/RTCP as network transmission protocol, to better meet the realtime requirements of streaming media transmitting. It also has some other key network communication features, such as supporting IPv4/IPv6 dual stack, NAT traversing, communicationsecurity protection, adaptive bandwidth control and error resilience.The terminal can work under both IPv4 and IPv6 network, configurate network interface through DHCP, detect the presence and behaviour of the NAT and traverse it. User can dial a SIP number, and then connect to another terminal by SIP signaling. After the session is created, the compressed video and audio streams will be transmitted by RTP/RTCP protocol. It can also store video and audio streams to harddisk or send/ broadcast/ multicast them to the network, as a video surveillance terminal do. Besides these, the terminal can play back local historical record files saved in its harddisk, or movies downloaded from the VOD server.The process of the terminal design and realization can be divided into following steps.(1) Using VM600 processor, flash, DDR SDRAM, OV9650, AK8812, W681310, expanded IDE interface to design hardware circuit and ultimately the hardware platform of the terminal is implemented. VM600 is a high-performance multimedia processor introduced by VIVO Micro Corporation. It integrates two MIPS cores working at 200MHz, a hardware MPEG-4 encoder/decoder engine, and many other peripherals. OV9650 is a digital camera used for video capture, and AK8817 is a TV encoder used for video output. W681310 is an audio codec used for audio recording and playing. IDE interface realizes high-capacity data storage. Terminal uses Flash memory (S29GL064A) of 8M bytes and DDR SDRAM (NT5DS16M16CS) of 32M bytes to storage data, and also affords space for running programs.(2) Make cross-compile toolchains such as binutils, gcc and uclibc for little-endian MIPS32 architecture. Then implement a JTAG Flash programming tool, which will program the bootloader into flash, under MFC framework using Visual C++. U-boot 1.1.4 is ported as bootloader of the terminal. uC/OS-II 2.52 and Linux 2.6.19 are also ported to support realtime multimedia processing tasks and the communication appilication. According to hardware interface and time sequencing, drivers of every external reference are modified and finally finished based on actual hardware circuit. By planning flow of the system booting, cross-compiling corresponding dynamic libraries and writing the booting script, file systems with Squashfs and JFFS2 formats are realized ultimately.(3) Analyze the shared memory communication mechanism; the flag interrupt asynchronous notification method and the semaphore synchronous primitive. Then implement the inter-CPU communication mechanism between MIPS cores, by defining a message queue.(4) Tasks running under uC/OS-II related with inputting, encoding, decoding and outputting of video and audio streams are designed, with which the terminal application may access multimedia streams through a series of well-defined APIs. The decoding task also solves the streaming-media synchronization by time stamp with a jitter buffer.(5) A SIP/SDP protocol stack for session initializing and media format describing and a RTP/RTCP protocol stack for realtime multimedia streams'transmitting are also realized. According to these, the communication terminal application is designed. It divides the whole media streaming and processing flow into several steps, such as media frames reading, network sending, disk saving, network receiving, media frames presenting, etc. Each of these steps is contained within a filter object. These filters have inputs and/or outputs, and can be connected together to become filter chain. With this mechanism, the terminal application can be a video phone, a video surveillance terminal or a VOD client, just by connecting these filters in different ways.Function testing is carried out with several other kinds of SIP user agent in both IPv6/IPv4 LAN and IPv4 with NAT public internet network environment. Result indicates that the terminal can encode, send, receive, decode and play video and audio streams persistently. Video streaming is played smoothly and the audio streaming is synchronized favourably. The terminal can be expanded both on software and hardware in order to be applied in videoconference, video wi-fi phone and etc.