| Vision supplies80%percent of information which human can perceive from the world, and image as the major expression form of the vision information has become the basic element and vector for human to recognize and comprehend our world. However, the methods which we have to tackle and exploit the images are not satisfied the requirements of the realistic applications. This status does not only waste the image sources but also restrict the progress in utilization of images. Especially, following with the increasingly progress and improvement of the sensor technology, the automatically data collection technology and the computer technology, there are some new opportunities and challenges for treating, analyzing and comprehending the images. The earth is moving continuously, consequently, how to get the accurate moving information from the image sequences is crucially importance for us recognize the world. How to analyze image sequences (e.g. motion estimation and motion feature detection) is a basic and difficult issue between the low level image processing and high level image analysis and image comprehension. The optical flow method provides a possible way to solve the above problem, and open the door of wisdom for the human.In the past more than30years, various optical methods are proposed and occurred. Due to the following advantages:1) Completely mathematical theory;2) Getting accurate dense motion field which has high spatial-temporal resolution;3) Easy for parallel computing and implementation on hardware devices, the variational optical method has become one of the most successfully optical flow approaches. However, the two constraints-brightness constant assumption and the smoothness assumption of the variational optical flow equation, resulting in it generates huge bias from the actuality when meet with complicated cases. For instance, noise, occlusion, large displacement, illumination changes, multiple motion objects, and non-rigid motion, etc. Hence, we need to propose new ideas and approaches to deal with these complex conditions, to construct the more effective variational optical models which can be universally used.In this paper, we do some research to treat with the above mentioned difficulties by proposing new techniques and modifying the variational models. Majorly focusing on how to improve the robustness of the variational optical flow model to strengthen its ability to treat with noise, occlusion, large displacement and illumination changes. etc. Especially, improving the motion discontinuity preservation performance of the variational model. We also propose a useful approach to automatically select the optimal smoothness parameter λ to keep the balance between the data term and the smoothness term. Finally, we do some work to extract the motion features of the optical flow field. The concrete works can be summarized as below:(1) We analyze the functions and performances of various effective optical flow techniques. Because of the viriational model has the advantage that it can combine different concepts and approaches within a single energy functional, and thus the performance of the combined model can be enhanced by merging the merits of these different available concepts and approaches. Based on this characteristic, we select the advanced TV-L1-Duality model as the foundation, and than choose some suitable techniques for combination. Such as. the median filter technique which can be used for post-denoise the optical flow field; the Coarse-to-Fine technique which can deal with large displacement and also enhance the computational efficiency; the Structure-Texture decomposition technique which is good at handling illumination changes; the robust function which is robust to outliers and can deal with multiple motion objects; the highly accurate image interpolation method and the blended image gradient method which can reduce discrete errors, etc. The combined TV-L1-Duality optical flow model can handle almost all the complex cases, widening the effectiveness of variational optical flow method.(2) Based on the basic feature of motion, we propose an optical flow vector availability detection strategy. This strategy is derived from two motion characteristics:1) motion linearity;2) motion uniqueness. The large bias and the error optical flow vectors can be detected by these two characteristics. The multiple motions and occlusions, etc, will give rise to motion discontinuities. And in the motion discontinuities area, the basic assumptions of the variational optical flow equation are disturbed, hence, amount of biases and errors are generated. Obviously, these bias and error optical flow vectors are not satisfied the motion linearity and motion uniqueness. Consequently, they can be easily detected with our strategy, and then we label them.(3) Based on the locally similarity of the optical flow field, we propose a neighborhood correction technique to remedy these detected inaccurate optical flow vectors. There is no one point and all its neighbors are wrong in the estimated flow field. In other words, at least one neighbor of problematic point is right. Due to this feature, we present a weight approximation technique. Distributing the weight to the right neighbors, and than combining these neighbors to form one new value to replace the problematic point. With the correction, the bias and error optical flow vectors around the motion discontinuities area are reduced, and the accuracy of the estimated optical flow field are improved. Most importantly, the motion discontinuities are well preserved.(4) The smoothness weight A plays an important role in controlling the trade-off between the data term and the smoothness term. If A is too small, it will result in overfitting between the two frames. If λ is too large, the flow field would be too smooth, some motion information could be smooth out. Whether the selected smoothness parameter λis optimal in the numerical computation process can directly affect the quality of the flow field. We introduce the RMS (Root Mean Square error) criterion which are widely used in block matching performance evaluation, to the variational optical flow model as an evaluation measure to automatically determine the optimal smoothness parameter λ. In order to improve the effectiveness of the RMS, we use the gradient of difference image (the difference between the interpolated warp image and the sample image) as the weight to combine with RMS. and construct a WRMS criterion. The experiments demonstrate that the WRMS criterion is good at determining the optimal smoothness parameter A automatically. Based on WRMS. we form a new variational optical flow model: λ-WMF-TV-L1-Duality. This new model is very useful for selecting optimal smoothness parameter λ to achieve good trade-off between the data term and the smoothness term, and the accuracy of optical flow filed is greatly enhanced.(5) Because the features of the optical flow field nearly contains all the important motion information of the image sequences, thus accurately detecting the features of the motion field is crucial importance. Secondly, as most of the motion errors are distributed at the motion feature areas, hence, suitably post-processing of the feature areas would further reduce the inaccurate flow vectors and improving accuracy. In this paper, we analyze the different advantages of the advanced Weighted Median Filter and the Bilateral Filter, and present a technique to use both of the two filtering methods distinguished. As the Bilateral Filter is more suitable for the occlusion area, we classify the features areas into occlusion boundary areas and the general edge areas. Experiments show that the post-distinguished filtering technique can effectively reduced error flows around the feature areas. Especially, some occluded region, which cannot obtain motion information without the post-processing technique can regain approximate right motion information due the Bilateral Filter. This post-distinguished filtering technique also improves the accuracy of the estimated optical flow field. |
PDF Full Text Request