The Design And Implement Of Audio-Video Server Software Based On The Chip Of Hisilicon Hi3510

The Audio-Video Server is an interdisciplinary application field emerging research areas such as multimedia processing technology,networking communication, artificial intelligence,etc.With the rapid development of multimedia processing technology and other related technologies, the Audio-Video Server itself progresses greatly. The first generation of Audio-Video Server is constructed on tape-based platform and processes analog-signals. Now, the latest Audio-Video Server is transplanted to high performance computer-based and networking platform concentrates on digital signals.An embedded system consists of an embedded processor and an embedded operating system. Embedded operating system runs on embedded processor, it can dispatch all tasks, manage the processor and other hardware devices of bottom layer. Linux is one kind of general embedded operating system, easy to transplant, formidable function, has a very good affinity to the network.The Hi3510 chip which the HISILICON Company promotes is a video compression chip based on H.264 BP algorithm.The chip is polynuclear high integration SoC integrates ARM+DSP+hardware acceleration engine, has formidable video processing function.Linux runs on the ARM processor, realizes the network communication and the terminal application procedure; DSP and the hardware acceleration engine completes the function of media real-time processing.In this thesis, we present a brand-new solution for Audio-Video Server which using the chip of HISILICON Hi3510.The Audio-Video Server is designed based on Hi3510 and Linux 2.6.14 operating system. It adopts RTSP protocol to initialize sessions with client terminals and describe the media format, and adopts RTP/RTCP as network transmission protocol, to better meet the realtime requirements of streaming media transmitting. It also has some other key network communication features, such as supporting IPv4 stack, communication security protection, and adaptive bandwidth control and error resilience.The Audio-Video Server can work under IPv4 network. After the user using client side software starts the connection function, the session which connected with sever is created by RTSP protocol. The compressed video and audio streams will be transmitted by RTP/RTCP protocol. The user using client software can store real-time video and audio streams to local harddisk. Besides these, the client software can also play back historical record files saved in local harddiskThe process of Audio-Video Server design and realization can be divided into following steps.1) Using Hi3510, flash, SDRAM, SDC-413, ADV7171, TLV320AIC23B, and TW2834 to design hardware circuit and ultimately implement the hardware platform of the Audio-Video Server. The Hi3510 is a high performance embedded multimedia processing chip which the HISILICON Company promotes. The chip with polynuclear high integration SoC structural integrates ARM+DSP+hardware acceleration engine, can complete the media real-time processing, the network service and the terminal application procedure perfect. SDC-413 is a digital camera used for video capture, which supports the D1 form of video signal input. TW2834 supports four channels in one channel out function and other video processing function .ADV7171 is a TV encoder used for video output, which converts the Hi3510 output digital video signal form of CCIR BT.656 to analog signal. TLV320AIC23B is an audio codec used for audio recording and playing. The Audio-Video Server uses Flash memory (S29GL256N10TF) of 16M bytes and SDRAM (MT48LC16M16A2) of 32M bytes to storage data, and also affords space for running programs.2) U-boot 1.1.4 with its FLASH,SDRAM,Ethernet part been modified to fit the hardware , is ported as boot loader of the terminal. Linux 2.6.14 are also ported to support real-time multimedia processing tasks and the communication applications. Based on hardware interface, time sequencing and actual hardware circuit, drivers of every external reference are added and modified. Planning flow of the system booting, cross-compiling corresponding dynamic libraries and writing the booting script, modify NFS net file systems, make cross-compile environment. Complete the conformation of the Audio-Video Sever software platform.3) Audio-Video Server's application software uses structure of multithreading, realizes the reception of surveillance data, network transmission and user control function. Use data queue and mutex mechanism between the threads to realize communication and synchronization. Use application program interface (API) function program the real-time media processing software runs on the Linux operating system. According to various functions, designed audio module, video module, compo sited audio and video module and application module. Each module can be redesigned into different functional sub modules, such as video module can be redesigned into video input module, video coding module, video output module. Each module defines input and output interface, through these module we achieve audio the video server basic functions such as audio and video data acquisition, encoding, decoding.4) RTSP protocol stack for session initializing and media format describing has been realized in Linux operation system. RTP/RTCP protocol stack for real-time multimedia streams'transmitting and feeding back network information are also realized. The multimedia communication terminal application based on Linux is designed. It divides the whole media streaming and processing flow into several modules, such as media frames reading, media frames presenting, network sending, etc. Each of these functions is contained within a module, which is independent from each other. These modules have inputs and/or outputs interface, and can be connected together to become modules chain to realize some function. In this mechanism, we can realize the audio video server function of transmitting stream by connecting these modules.Function testing is carried out with audio video server and several PCs which have installed client software in both IPv4 LAN and IPv4 public internet network environment. The result indicates that the server perform four channels NTSC/PAL CIF mode video streams'real-time collection, H.264 compressing, network transmitting (or one channel D1 mode real-time).And also perform four channels audio streams'real-time collection, compressing, network transmitting. The client software performances four channels real time H.264 video streams' and audio streams' network receiving, decoding replay, the video streams are fluency, clear and stabilization, audio streams are smoothly and fluency, video and audio streams' synchronization is well, they are well in matching. The system also supports pan&tilt controlling, joint alarming, user purview validating, dynamic parameter modification etc.