Segmentation and matching of moving vehicles from complex outdoor scenes

This thesis describes a machine vision system which is able to extract moving vehicles from complex stationary backgrounds and match them based on color and shape information. This vision system was developed in support of the Intelligent Vehicle/Highway Systems (IVHS) program whose main goals are to reduce the number of accidents on the road, increase the traffic density, and provide route guidance to the travelers. The specific problem of interest in this thesis is to estimate the average travel time between two points in a road network.; We propose an object matching system which includes a number of modules that can be utilized in other application domains. In particular, we have developed four image segmentation algorithms. The motion segmentation algorithm can identify the moving areas in the image using a three-frame sequence. The color segmentation algorithm combines edge information and a split-and-merge technique to identify homogeneous regions of different colors in a color image. These two algorithms are general purpose image segmentation algorithms.; The other two segmentation algorithms utilize object model information. We propose an algorithm to extract an accurate contour of the moving object in the image. This technique fuses color and motion information using the theory of active contours to constrain the contour of the moving object to be smooth. The final image segmentation algorithm constrains the object to be a vehicle. We define a generic model of a vehicle and use the theory of deformable templates to fuse motion and edge information and obtain a segmentation and classification of the vehicle of interest.; Once the vehicles of interest have been separated from the stationary background and the other moving vehicles, the matching strategy is based on color and shape matching. For color matching, we compare the 3D color histograms of the two objects. For shape matching, we propose a modified Hausdorff distance between point sets to compare the edge images of the vehicles.; We evaluated the performance of the vision-based matching system on a database of 287 vehicles. We were able to segment and correctly classify 91.6% of the vehicles. We also demonstrated the efficacy of time-based, vehicle class-based, and color-based indexing schemes. We compared the machine vision results to the results that experts obtained manually by looking at the video tapes and concluded that the machine vision system is a feasible tool for travel time estimation.