Research On The Extraction Of Salient Object Based On Egocentric Video

Salient object extraction based on egocentric video means extracting significant object regions with independent visual significance in video,which is of great importance to advanced visual tasks such as action recognition,scene understanding and semantic analysis.In recent years,with the extensive application of wearable life recording cameras,salient object extraction based on egocentric video has attracted wide attention of researchers.Based on the analysis of egocentric video,such as a large amount of redundant information and significant differences in object scales,it is of great significance to study salient object extraction based on egocentric video.In this paper,egocentric video sequence is taken as an input to study the meaningful frame extraction algorithm and the salient object extraction algorithm based on spatio-temporal saliency,finally,an egocentric video summary focus on meaningful object is generated.The main work of this paper is as follows:Firstly,we study the algorithm of meaningful frame selection for egocentric video based on CNN feature.Most of the current training data in video field is fixed view video data,training data and test data(egocentric video data)disobey the same distribution.Traditional machine learning assumes that the training data and the test data are subject to the same distribution,which means that we need to annotate a large number of egocentric video training data.Aiming at this problem,this paper proposes a domain-adapted mcaningful frame extraction algorithm based on CNN feature.This method uses the fixed view video data which has been marked as meaningful frames as the source samples,extracting the CNN feature of source domain and target domain(egocentric video)respectively to construct the domain invariant feature space which connects the source and target domains.Similarity between the source sample and the target sample is calculated in the domain invariant space,and meaningful frames in the egocentric video are extracted according to the confidence of the meaningful frame.The comparison experiment on UTEgocentric dataset validates the effectiveness of the proposed algorithm.Secondly,we study the algorithm of salient object extraction based on spatio-temporal saliency.This paper proposes a progressive salient object extraction algorithm based on spatio-temporal saliency.Firstly,the initial object segmentation result is obtained by using temporal saliency,which is the candidate region,and then the spatial saliency of the object is calculated from the pixel and the region respectively in the candidate region,spatio-temporal saliency map of egocentric video is obtained by integrating temporal saliency and spatial saliency information,finally,salient object is extracted based on saliency map.The object detection experiments based on saliency show that the algorithm proposed in this paper is better than IT,GB,SR,CA and AGV algorithms for the saliency detection of egocentric video.The comparison between the experiment result of salient object extraction based on spatio-temporal saliency map verifies the effectiveness of the proposed algorithm.Finally,we propose a method for egocentric video summary focus on meaningful object.Given a sequence of unedited egocentric video frames,we extract the meaningful frames based on the algorithm proposed in chapter two.Based on the obtained meaningful frames,we extract the salient object and obtain the meaningful object video frame.Then the meaningful object video frames are sorted by time,the number of object categories in the video is obtained by performing cluster recognition on the object,and finally an Egocentric video summary focus on meaningful object is generated.