Inertial Constrained Hierarchical Belief Propagation For Optical Flow

Optical flow estimation is a basic research area in computer vision.It is widely used for moving detection,object segmentation,tracking,scene classification and many other applications.After decades of study,researchers have proposed many models to calculate optical flow.But up to now,estimating accurate large displacement optical flow remains a challenge for most models.Recently,most researches focus on combining the matching algorithms with variational model to overcome the drawback of conventional variational model in optical flow estimation.As a method for finding correspondences between images,loopy belief propagation(LBP)has been more and more widely applied in optical flow estimation.But its efficiency is capped by huge consumption of memory and low accuracy when facing large displacements.In order to improve the accuracy and speed of loopy belief propagation in optical flow estimation,we propose an inertial constrained hierarchical belief propagation method for estimating optical flow accurately.We segment the image into superpixels and construct MRFs on superpixels and pixels respectively.Every possible displacement is a label.Since a superpixel covers an area,we enlarge the step of labels so that label space can be relatively small when covering a large displacement scope.We first obtain a basic displacement field by performing LBP on the superpixel MRF.It is used as a reference when choosing labels on the pixel MRF,which can effectively compress the label space and accelerate the process of LBP on the pixel MRF.In scenes where large displacements occur and not enough texture information can be provided,conventional two-frame model cannot obtain reliable results.In order to solve this problem,we integrate multi-frame image information and previous displacement information as inertial constraint into the proposed model.The proposed constraint can make up for lost information and improve our model's performance.The proposed model obtains competitive results on KITTI dataset and MPI Sintel dataset,which show the effectiveness of our model.