Optical-Flow-Guided Multi-Keyframes Feature Propagation And Aggregation For Video Obiect Detection

Recently,with the development of deep learning,deep convolution neural networks have achieved great success on image recognition.The success promoted the extension and expansion of image recognition technology to the video field.Object detection is the most challenging problem in the field of computer vision.Now,the algorithms for image object detection problem are relatively mature and effective.However,there are few studies on video object detection.The video contains more information than the image.With the development of communication technology and computer technology,the number of videos in the network is increasing.Video object detection has become one of the research hotspots in recent years.Based on DFF algorithm and FGFA algorithm,this paper presents a new algorithm,called MKP algorithm,which combines the advantages of DFF algorithm and FGFA algorithm.The core idea of MKP algorithm is Multi-Keyframes Feature Propagation,and its principle is to select m as the key frames from every n frames,then the feature of key frames is calculated by the feature extraction network,and the feature of non-key frames is obtained by aggregating the propagation feature from key frames to the current frame.Experiments show that we can get models with different accuracy and rate by adjusting the value of m/n,so that the MKP algorithm can be applied to different scenarios and needs.The feature aggregation of MKP algorithm is linear,whose core point is the determination of the weigth.The paper proposes two aggregation weights,one is one-dimensional relative distance weight,and the another is two-dimensional cosine similatity weight.The paper also proposes a SISK optimization strategy for the MKP algorithm.Experiments show that the SISK strategy can improve the accuracy of the MKP algorithm by about 0.32 percentage points,and has no effect on the rate.