Research On Streaming Media Synchronization In Wireless Multimedia Sensor Networks

Internet of Things (IoT) has become not only the focus of the national competition, but also one of the most important infrastructures of the economic development, social progress and technological innovation. This technology will promote the interconnection and fusion between the physical world and information space. Wireless Multimedia Sensor Network (WMSN) is an important part of the Internet of Things. It has the characteristics of rich information and flexible deployment. In WMSN, the multimedia synchronization is the basis for the in-network collaborative processing and provides the important foundation for the WMSN applications. Simple data synchronization in traditional Wireless Sensor Network (WSN) usually employs the timestamp and requires the global clock synchronization. However, for WMSN, the introduced streaming media has the features of abundant data, strong time correlation, real-time requirement, and so on. Traditional synchronization scheme can not satify the synchronization requirement of WMSN applications.For the streaming media synchronization in WMSN, in this paper, we study the special features (low bandwidth, limited energy resource, large block of the multimedia stream, etc.) of WMSN and propose a series of methods from the three aspects:data collection, transmission and reception. The basic ideas of this thesis are:first, synchronize the multimedia units precisely by employing the media stream correlation; then perform the active synchronization control during the stream transmission to balance the synchronization quality parameters; finally, construct the multi-level buffer, execute the synchronization control in the receiver and provide the smooth multimeida streams for the upper layer applications. The main contributions of this thesis are as follows:(1) For the data source synchronization in WMSN, firstly, we propose a distributed audio synchronization algorithm with the audio fingerprint. The synchronization task is shared by the cluster head and all the reporting sensors. Comparing with the traditional TDOA scheme, the proposed algorithm can reduce the energy consumption of the cluster head effectively, and relieve the effect of the network packet loss on the synchronization accuracy. On the other hand, for the data source synchronization of the audio and video stream, we present a heterogeneous network clock synchronization algorithm with the quick matching table. We construct a dynamic and fast clock matching table in the gateway by employing the minimum perfect hash function. We synchronize audio and video streams by searching the matching table. Experimental results show that this algorithm can synchronize the audio and video stream precisely with little effect of the dynamic network payload.(2) For the transmission synchronization in WMSN, we propose an active multimedia stream synchronization control scheme. This scheme includes three aspects: the jitter detection algortihm, dynamic priority scheduling algorithm and packet dropping mechanism with the data dependency. The jitter detection algorithm can detect and discard the jitter-corrupted packet immediately and maintain a high bandwidth for packets within the multimedia streams jitter tolerance. The dynamic priority scheduling algorithm can adjust the multimedia unit priority adaptively during the stream transmission, which help to reduce the average delay variation of the stream and decrease the packet loss caused by the jitter. The packet dropping mechanism with data dependency actively selects the packets to discard, relieve the effect of the packet loss on the synchronization quality during the transmission.(3) For the reception synchronization in WMSN, we propose an adaptive synchronization control scheme with the multi-level buffer. By combining the sensors in the stream transmission path, we manage the memory resource globally and build a distributed synchronization controller, which can solve the resource constraint problem of the WMSN sensor. The synchronizatin control scheme consists of two aspects:the buffer level adaption algorithm and packet scheduling scheme. The buffer level adaption algorithm can evaluate the current network state in real-time and balance the delay and jitter dynamically. The packet scheduling scheme can analyze and evaluate the buffer state in each level and relieve the effect of the jitter on the performance parameters of delay and packet loss.(4) For the system verification, we build a smart surveillance system prototype based on the WMSN. In the system, multimedia synchronization is the foundation of the in-network data fusion, redundant information filtering and real-time presentation of the multimedia streams. Based on the proposed streaming media synchronization algorithms, we construct a multimedia sychronization middleware. We test and evaluate the performance of these proposed synchronization algorithms in a real scene. The experimental results show that our proposed approaches can synchronize the media streams precisely and balance the synchronization quality parameters effectively.