Research On Optical Flow Prediction Algorithm Based On Improved Convolutional Neural Network

Optical flow is currently an important method and tool for motion image analysis,and has now developed into an important branch of computer vision.The optical flow represents a moving object in space,and the movement state of pixels in the observation reference plane.Optical flow prediction algorithm is to use the correlation and corresponding relationship between two adjacent frames in the image sequence to find the change of corresponding pixels in time and space to calculate the motion state of the target object between the two adjacent frames.The optical flow expresses the change of image pixels.Because it contains information about the movement of the target,it can be used by the observer to determine the movement of the target.The purpose of studying the optical flow field is to approximate the motion field that cannot be obtained directly from the image sequence.Therefore,the study of optical flow information has important theoretical value in the field of image processing.The prediction method based on variational energy model and the heuristic method based on block matching are the most mainstream traditional prediction methods in optical flow prediction,and these methods have limited adaptability to images,lack of generalization,low efficiency,and operation and calculation.It is more complicated and is not conducive to the extraction of optical flow of complex image data.The use of Convolutional Neural Network(CNN)can avoid the problem of cumbersome calculation and poor adaptability caused by artificially constructed features.By adaptively learning the required features through massive data,it can better extract the deep information of the data.Type image data has strong adaptability.However,because there are many complicated problems in image optical flow prediction,such as occlusion and large displacement,the target search islost and the image detail feature extraction is inaccurate.The traditional structure of the convolutional neural network cannot solve these problems properly.Therefore,in order to solve the above complex problems and take into account the universality of the algorithm,this paper studies the convolutional network to solve the problem of optical flow prediction,mainly including the following work:(1)In-depth analysis and research on the feature extraction process and principles of optical flow prediction methods.In order to solve the large displacement and image detail problems in optical flow prediction,the feature extraction part of the existing convolutional neural network is improved.The feature extraction part is mainly composed of multiple convolutional layers.In this paper,the first layer of the network is changed to a convolutional layer composed of Deformable Convolution and Deformable Pooling with stronger adaptability,Which improves the network's ability to adaptively optimize the image,which is beneficial to capture the details of the motion contour,and at the same time,it has a stronger adaptability to the displacement of the pixel size between frames,especially in the case of large displacement,it can provide a larger receptive field.Capture the movement of pixels.The experiment proves that the network with deformable convolutional layer can better extract the image detail features of adjacent frames,and at the same time improve the ability to capture large displacements.(2)To solve the occlusion problem and improve the prediction ability of the optical flow prediction network,the key is to calculate the correlation between two adjacent frames from deep spatial features,and based on the process characteristics of the optical flow prediction network.The fusion part starts with improving the matching fusion mechanism commonly used in existing optical flow prediction models based on convolutional neural networks,and introducing a feature correlation layer based on attention mechanism.Superimpose the deep-level features of two adjacent frames channel by channel to reconstruct the channel dimensions to maximize the preservation of the effective image spatial features while calculating the correlation between the two parts of the features so that subsequent deconvolution operations can better predict Provide precise and clear optical flow.Experiments show that the above algorithm not only effectively improves the clarity of optical flow prediction,but also greatly improves the occlusion problem.(3)While addressing the problems of occlusion,large displacement,andimage detail presentation of optical flow prediction as mentioned above,in order to ensure the universality of the algorithm,loop optimization and network stacking strategies are introduced.The main principle is to cascade several networks with different structures and characteristics to form a network stack,so that the optical flows output by each sub-network are optimized through multiple network cycles and then combined together to improve the accuracy of optical flow prediction..The network stack in this paper sets up three kinds of sub-networks with different structures and internal modules,so that the network can combine the advantages and characteristics of different sub-networks and output the final optimization results.(4)In order to verify the rationality and superiority of the algorithm in this paper,the proposed optical flow prediction algorithm based on convolutional neural network is trained and experimented on the common data set Flying Chairs and Mpi Sintel,and the current mainstream optical flow The accuracy comparison of the algorithm shows that the proposed algorithm has improved the optical flow accuracy.In order to fully verify the targeted performance of the improved algorithm in occlusion,large displacement and image detail presentation,the corresponding image is selected from the data set and compared with the mainstream algorithm.On the one hand,comparing the experimental data,the results show that,on the Flying Chairs dataset,the average end point error of the predicted optical flow and the actual optical flow of this algorithm is 1.75,which is lower than the comparison algorithm;in the Mpi Sintel dataset,the algorithm of this paper The average end point error is 3.83 and 1.285,which is also lower than the comparison algorithm.On the other hand,comparing intuitive images,the results show that the proposed method is also significantly better than the comparison algorithm for the prediction results of samples with problems such as occlusion,large displacement,and image detail presentation.In summary,the model in this paper has higher accuracy and robustness to complex problems such as occlusion,large displacement,and image detail rendering.It also proves that the use of deformable convolution and association layers based on attention mechanism to improve convolution The important role of neural networks in solving such problems.