The Research On Object Correspondence Technology Based On Geometric Constraint In Multi-camera Video Surveillance System

In recent years, with the increased prominence of public safety, the innovation of high-performance micro-processors and the continuous development of video analysis technology, video surveillance system has been widely used in many industries, such as security, transportation and medical treatment. However, the use of single-camera is limited in monitorable scope, difficult to resolve the object occlusion problem and hard to keep continuous tracking of the object cross multi-camera. By collaboration between multiple cameras, it has the advantage of expanding the field of view, increasing the observation angle, capturing more comprehensive information of the object. It is also help for coping with object occlusion, environmental illumination change, 3D object modeling and so on. For this reason, intelligent video surveillance system based on multi-camera collaboration has attracted wide concern among researchers, which is regarded as a hot and difficult topic.The object correspondence as the key technology in intelligent video surveillance system based on multi-camera collaboration is studied in this thesis. Object correspondence between multiple cameras involves at the same time instant finding correspondence between objects in the different image sequences and giving the same label. In this thesis, geometric constraint relationship containing epipolar geometry constraint relationship and homography constraint relationship is adopted. The main contributions and innovation points are as follows:(1) In order to get the epipolar geometry constraint relationship, a new and robust estimation method for fundamental matrix is proposed. Firstly, the SIFT algorithm is adopted to extract and match feature points, with some pretreatment to get the initial matching point collection. Then, more accurate matching point collection can obtain by combining LMedS method with M-estimators method to remove mismatching points. Finally, the Levenberg-Marquardt nonlinear optimization algorithm is adopted to obtain more accurate estimation of fundamental matrix. The experimental results show that: the proposed method is robust and highly accurate.(2) In order to get the homography constraint relationship, for inaccurate coordinates which is used for estimation for homography matrix, a simple two-step method for obtaining relatively precise coordinates based on epipolar geometry constraint relationship is proposed. Firstly, initial coordinates can be obtained by manual method which takes the transfer error as the criteria. Then, more precise coordinates can be obtained by optimal method which takes the sum of epipolar distance as the criteria.(3) For the LMedS-based principal axis extraction method which is time-consuming and hard to cope with the condition that the camera has great tilt angle, a new method based on K-L expansion is proposed to extract the principal axis of the object. Firstly, the proposed method is simply to calculate the second-order central moments of the binary image. Then, the second-order central moments are calculated to determine the two parameters of the principal axis: the centroid and the angle a with respect to the up border of the object.(4) Based on the above research, a new object correspondence method based on principal axis and epipolar geometry constraint relationship is proposed (PA-EGC:Principal Axis and Epipolar Geometry Constraint). Firstly, object correspondence based on principal axis is accomplished. Then, due to the obtaining error of ground-point, the calculation error of intersection and so on, the object correspondence method based on principal axis may forget the correct object pairs in some condition. In order to make up this defect, a second-match process based on the epipolar geometry constraint relationship, which takes centroid as feature point, is proposed. The experimental results show that: in terms of the extraction time of principal axis, matching accuracy and applicability to different scene, the proposed method is better than the literature [35].Finally, the summary of this thesis is made, and future direction on the object correspondence is discussed as well based on the work of this study.