Research On Some Key Technologies Of Multi-cameras Panoramic Video Surveillance

Video surveillance has been one of the most active research areas in computer vision. The goal is to efficiently extract useful information from videos collected by surveillance cameras. The view of a single camera is finite and limited by scene structures. In some applications, such as tracking a vehicle traveling through the road network of a city, video streams from multiple cameras have to be used to collaborative surveillance. Multi-cameras video surveillance can not only monitor a wide area, but it also able to resolve some great intractable problems in single-camera video surveillance, e.g., objects occlusion or 3D reconstructions. However, at the same time, multi-cameras video surveillance also faces many new challenges, e.g., match the objects across camera views with large changes of viewpoints and illumination conditions, multi-cameras tracking with complex topology of a large camera network, it is urging in-depth research on techniques suit for multi-cameras video surveillance.The aim of this dissertation is to development a multi-cameras panoramic video surveillance system, some key technologies within the multi-cameras panoramic video surveillance system are thoroughly studied. The main achievements of the dissertation are as follows:(1): A robust feature point matching algorithm named spatial order constraints bilateral-neighbor vote(SOCBV) is proposed to remove outliers for a set of matches(including outliers) between two images. A directed k nearest neighbor(knn) graph of match sets is generated, and the problem of feature point matching is formulated as a binary discrimination problem. In the discrimination process, the class labeled matrix is built via the spatial order constraints defined on the edges that connect a point to its knn. Then, the posterior inlier class probability of each match is estimated with the knn density estimation and spatial order constraints. The vote of each match is determined by averaging all posterior class probabilities that originate from its associative inliers set and is used for removing outliers. The algorithm iteratively removes outliers from the directed graph and recomputes the votes until the stopping condition is satisfied. Compared with other popular algorithms, such as RANSAC, RSOC, GTM, SOC and WGTM, experiments under various testing data sets demonstrate strong robustness for the SOCBV algorithm.(2): An automatic local multi-cameras color correction algorithm is proposed to solve the color inconsistency problem of the local multi-cameras system.Firstly,the color correction parameters of adjacent cameras have been obtained using the invariance shape of color histogram in the overlapping between cameras. Then,in order to minimum the error caused by parameter passing, the reference camera is selected by using defined measurement function of reference camera, and the parameter passing path of each camera is optimized by Single Source Shortest Path Algorithm.Finally,the color correction parameters of cameras have been obtained by parameter passing.The experimental results show that the proposed algorithm can automatically realize the local multi-cameras color correction, and effectively solve the color difference problems caused by cameras.(3): A stable color texture description algorithm named local constancy color local binary pattern(LCCLBP) is proposed to improve the robustness of the existing color local binary pattern algorithms. For a color vector in an image, firstly, the corresponding local constancy color vector has been computed based on the assumption of local color constancy. Then, the local constancy color component of the color vector is obtained by vector decompose with local constancy color vector. Finally, the code of LCCLBP is computed from the order relationships among local constancy color components. Compared with other popular color LBP algorithms under various testing data sets, the LCCLBP algorithm not only can effectively describe the local color texture, but also has strong robustness for the changes of illumination conditions and cameras.(4): An efficient effective Mean Shift clustering algorithm named bilateral adaptive mean fusion shift(BAMFS) is proposed to address the problem of the traditional Mean Shift optimization methods. First, the density relationships between sample points and estimate point under different adaptive bandwidth Mean Shift methods have been analysed. Then, a negative Mean Shift optimization method is proposed to ensure the Mean Shift vector is away from the density descent direction. Finally, the BAMFS algorithm combining negative Mean Shift optimization method and positive Mean Shift optimization method is proposed to improve the ability to escape from the local maximum destiny. Experimental results show that the clustering performance of the proposed algorithm is better than that of the existing Mean Shift optimization methods.(5): A multi-cameras panoramic video surveillance experimental verification system is designed in this paper. The above achievements in the dissertation are applied into the system to verify the validity.