| Our project's goal is to do some stereo vision with pairs of images where one of the images is from the visible spectrum and the other is from the infrared spectrum. The reconstruction is limited to only a relative depth value for each moving object in the scene: a foreground blobs' horizontal disparity.; In the literature, we can find many camera calibration algorithms, feature points finders, pairing algorithms and reconstruction algorithms. In our case, both cameras are not calibrated. Furthermore, most classical techniques to create reliable pairs of points are not able to recognize a point of the visible image in the infrared image.; Our method is based on the background subtraction of each image individually. All foreground blobs become an invariant source of information according to the type of image used. From there, we have two methods: the skeleton method and the DCE method. The skeleton method consists in finding in the skeleton the local axes of symmetry of the shape. Then, we analyze the intersections of the axes having one, three or more edges. The orientation of the intersection points' edges and the minimum distance between the intersection points and the blobs' contour are part of the information used to create the pairs of points. The method based on the DCE process consists in simplifying the blob's contour in order to keep only a fixed number of significant vertices. Again, the orientation of the two edges of the chosen vertices and the relevance measure of these vertices in the approximated contour are used to create the pairs of points. The pairing process is done with a correspondence matrix where each point in a pair has chosen the other mutually.; In our system, we apply two successive filters to eliminate the outliers in the pairs of points: the use of blob pairs and the use of a RANSAC algorithm that models inliers with a fundamental matrix. This last method is largely used and documented in the literature.; The final result of our project is the median horizontal disparity of each found pair of blobs, i.e. a relative depth value for all elements of the scene that are not in the background. We compare this horizontal disparity with the disparities that are in ground truths. In the comparison, we use a score calculating the absolute difference between each computed disparity and its known corresponding disparity in the ground truth. This difference is then normalized according to the length of each disparity (the calculated one and the ground truth one). For each pair of images, we retain 1.0 minus the mean of the scores of all valid pairs of blobs. The final scores are from 0.0 to 1.0, where 1.0 is a perfect score.; The test results show that our methods that find feature points in different types of image usually give final scores between 0.8 and 0.9 on 1.0. We have compared our methods with an adaptation of the method using the phase congruency: the numerical results (the final score) of our methods are slightly better than the results with the phase congruency. We have also used methods based on the minimal eigenvalue algorithm and on the Harris and Stephens operator: these methods are just not able to find good pairs of points in different types of image, mainly because they are generally not able to find the good pairs of blobs.; Finally, our final score criterion is too sensitive to evaluate properly small disparities. Consequently, the visual result may be better than the numerical result. So, more tests may prove numerically that our methods can identify the relative distance between each person in the scene and the order of these: from the closest to the farthest. |
