Research On Method Of Weakly Supervised Action Localization

Action localization has broad application prospects,and the development of fields such as autonomous driving,medical care,monitoring,and video retrieval relies on technology of action localization.However,traditional fully supervised methods of action localization require detailed labeling of video data,which are time-consuming and laborious,and are prone to errors.They are not conducive to the promotion of the method,and cannot process a large amount of video data on the network which are only partially labeled.In order to make the technology of action localization applicable to large data sets,the research on the weakly supervised action localization has great practical significance.Three weakly supervised spatio-temporal methods of action localization are proposed to adapt to different application scenarios.Firstly,a weakly supervised method of action localization based on template matching is proposed.The actual action position in the training video is no longer manually labeled.Instead,action templates are used to select the best action proposal from numerous action proposals as the position of the action in the video.This method only needs to label a small number of frames in the training set for action templates,and does not need to label every frame,which greatly reduces the workload of labeling data set.Then,in order to further reduce the workload of labeling data set,a weakly supervised method of action localization based on single frame matching and a weakly supervised method of action localization based on sparse points matching are proposed.The weakly supervised method of action localization based on single frame matching performs rectangular labeling on only one frame for each action instance in the training video.In the phase of training model,a new matching criterion is used to filter action proposals.The weakly supervised method of action localization based on sparse points matching labels some pixel points on partial frames of the training video to roughly indicate the spatial position of the action,and a new matching criterion isused to select the most suitable proposal as the position of the action in the training video.The weakly supervised method of action localization based on template matching only needs to label 2000 frames for the training set of UCF-sports.Compared with the fully supervised methods,the workload of labeling training set is reduced by69.7%.And compared with TLSVM,the accuracy of the action classification is increased by 0.3 percentage points.When the overlap threshold is 0.2,the accuracy of action localization is increased by 28.21 percentage points compared with CRANE.The experiment of weakly supervised method of action localization based on template matching on large dataset UCF101 requires manual labeling of 20200 frames,which is still very time consuming,and weakly supervised method of action localization based on single frame matching only needs to label 13320 frames,and weakly supervised method of action localization based on sparse points matching needs less annotation.Therefore,the latter two methods are more suitable for large data sets,but the accuracy of action localization is slightly worse than the weakly supervised method of action localization based on template matching.