Research On Key Techniques Of Optical Flow For Image Sequens And Its Application In 3D Reconstruction

Optical flow is widely used in the field of automatic robot navigation, pilotless automobile system, and facial expression dynamic tracking. Its ability to sense motion by vision is an important method to conduct target motion analysis and understanding. It has been an important research field in computer vision and related fields. A lot basic problems of optical flow algorithm have been resolved after 30 years of development. The application environment of optical flow has extended gradually from inside laboratory with ideal lighting conditions, artificial arrangement targets, to outdoor unknown environment with unknown targets. This extension means more and more optical flow algorithms had been applied in the practical projects. Meanwhile, the conventional optical flow algorithms have to face new challenges for the unknown environment and targets widely existed in practical application. These new challenges include:random variation of illumination in the outdoor, large displacement generated by small targets with a high-speed, occlusion between the targets, the self-checking for the correctness of optical flow computation and so on.This paper focuses on the key technologies of optical flow computation and its application in 3D reconstruction. Optical flow computation under random variation of illumination in the outdoor, self-checking for the correctness of optical flow computation and image matching with high precision and deleting redundancy 3D points in the process of 3D reconstruction were studied. The main work of this dissertation is as follows:(1)Because of the randomly variation of the outdoor illumination, optical flow algorithm must be able to adapt to the actual scene. The factors that affect the actual image acquisition process are analyzed based on the theory of light model and camera imaging models, and the illumination changes are divided into two categories. One is caused by the light that incident on the CCD(Charge Coupled Device) changes dramatically, and the other is caused by the relative motion between the camera and objects. The difference between histograms of two images were used to determine the type of illumination variety. If the illumination changes between two frames belongs to the first type, an improved Census transformation is used as data item of the optical flow model. Compared with the traditional Census transformation, the improved Census Transform contains the completed relative gray values among the central pixel to the neighbor pixels, which enable it to keep the same value when illumination changes monotonously, in the same time have more resolutions. If it’s the second type of illumination changes, the weighted sum of improved Census transform and texture features are used to construct the data items of optical flow model. The weight of each pixel is solved by the mean-field approximation theory. Compared to the fixed component of the data items, the proposed data term is more consistent with the law of illuminance variety in the image field. The proposed algorithm is tested on the database of Middlebury and KITTI, and the results showed the robustness to illumination change.(2)When the optical flow algorithm is applied to an practical project, certain accuracy need to be ensured. Currently, the major standard databases use the ground truth to assess the accuracy of optical flow algorithm. However, in practical engineering, the ground truth of optical flow and the targets are always unknown. Therefore, the ability to self assessment and self-correction for optical flow algorithm is a prerequisites for its application to real visual tasks. Aiming at this problem, according to the equivalent conversion between optical flow and image matching, an optical flow error assessment and correction algorithms has been proposed based on discrete reliable matching points. Firstly, image segmentation algorithm is used to segment the image into different regions, and each region expresses a small region of the rigid surface. Then, credible matching points in image will be detected. The credible matching points will be regarded as seed points, which will be used to assess and correct the optical flow error in the corresponding regions. Determining high precision credible match points with wide distribution and designing effective correction method are the key points of the proposed algorithm, a) To obtain quantity match points with high precision, an improved SIFT (Scale Invariant Feature transform) matching algorithm with two threshold between two frames is proposed. First, high precision matching points are calculated with a low threshold, and those matching points are used to calculate the pole geometry and spatial topology relationship between the two views. Then, the spatial geometry relationships are used as constraints to remove matching points with low precision calculated under a high threshold. b) In order to obtain matching points with wide distribution, a region matching algorithm is proposed based on HOG(Histogram of Oriented Gradient) feature and spatial geometry constraints. The inscribed quadrilateral of a region is regarded as a template, and the aim is to find the corresponding matching areas in the second frame. Epipolar and neighborhood constraints are used to reduce the searching ranges. To avoid the large matching error in the region with slightly changes in gray value, operators to measure the change of gray variation with region is designed. Only regions which meet a certain conditions are used to compute matching points, c) To ensure the effectiveness of the correction algorithm, criterion for piecewise-smooth optical flow in non-motion boundary was proposed, and epipolar constraint is used to carry out the secondary correction. The algorithm is tested on the KITTI database. Optical flow error is reduced by an average 7.04%, the maximum can be reduced by 31.25% on the KITTI database, which showed the effience of the proposed algorithm.(3)The optical flow algorithm is applied to 3D reconstruction for image sequences. For error matching, we use optical flow algorithm to compute the initial match points and then three-view geometric constraints is used to refine the matching points. For the increase of redundant points during the 3D reconstruction, a deleting redundant points algorithm is proposed. The algorithm can recognize whether the current reconstructed point was reconstructed in the previous views automatically. If it has been reconstructed, point with smaller re-projection error will be saved. Otherwise, the current point will be saved directly. The proposed algorithm can not only reduce the overall amount of data, but also improve the overall accuracy of point cloud without losing the details of the object surface. The proposed algorithm is also applied to active reconstruction system for deleting redundancy points. The experiments with the temple images verifies the effectiveness of the proposed algorithm.