Joint Estimation Of Binocular Depth And Optical Flow Based On Deep Learning

3D reconstruction is the core technology required in many fields,including robot navigation,target recognition,scene understanding,animation,industrial control,medical diagnosis,automatic driving and so on.Depth and optical flow are necessary for reconstructing real 3D scenes.Earlier methods were used to estimate depth or optical flow separately.In recent years,different methods for joint estimation of depth and optical flow have emerged.However,there are still shortcomings in generalization,accuracy and completeness,which seriously restricts the further development of this field.Therefore,in order to obtain more high-precision depth information and optical flow information,this paper proposes a new joint estimation method of binocular depth and optical flow.This paper establishes a unified network framework,uses unsupervised learning methods,and uses the attention mechanism methods to perform joint estimation of depth and optical flow,which obtains more high-precision estimation results.The main works are as follows: Firstly,a shared feature network based on the intra-task attention mechanism is proposed.A lightweight encoder-decoder backbone network is used for joint estimation.Skip-connection is designed to concatenate the encoder features and decoder features in the same scale.The multi-scale module is used to output the estimation results of multiple scales to train the loss function.And the intra-task attention mechanism extracts the inter-task features.Those initially improves the accuracy of the estimation.On this basis,a joint optimization network based on inter-task attention mechanism is proposed.Although the shared feature network uses the intra-task attention mechanism to learn the features of specific tasks from the global features,it does not interact with the task features.The joint optimization network uses the inter-task attention mechanism to fuse the multi-modal information of the intra-task features and generate intertask features,which can further improve the accuracy of joint estimation on the basis of the shared feature network results.This paper uses the road binocular videos of KITTI Dataset to train the network,and conducts experimental analysis through a series of self-comparison experiments,comparative experiments and visualized results,which proves the effectiveness of the proposed method.The detail results of low-texture or repetitive texture regions and the accuracy of joint estimation are improved.