The Research On Cascaded Action Detection Based On Boundary Probability

Action detection studies how to let the computer locates the action within the video,and classifies it into specific class.It is not only an important content of automatic video analysis,but also an essential research direction in artificial intelligence,human-computer interaction and autopilot.Most of the current action detection methods use an action proposal generation method to generate action candidate proposals first,and then adjust the candidate proposals to get the final detection results.Therefore,the performance of action detection method largely depends on the quality of candidate proposals.However,since the current action proposal generation method cannot fully capture the context information and does not consider the action correlation in time domain,the generated action proposals are often redundant in number and poor in quality.In order to solve the above problems,we present a Cascaded Boundary Network(CBN).The main work is as follows:(1)An action boundary probability prediction model based on temporal convolution neural network is proposed,which is used to locate the temporal boundaries with probabilities.This model not only has multi-scale receptive field,but also has adaptive receptive field,while still keeping a small amount of network parameters.This character allows us to capture the subtle changes of action in time domain and finally achieve accurate boundary prediction.(2)An action boundary refinement model based on Long Short Term Memory(LSTM)network is proposed,which is used to further refine the boundary probabilities obtained from the previous step.This model makes use of the memory characteristics of long shortterm memory network in time domain to capture the correlation between different stages of action,and further modifies the boundary probabilities.The main purpose of this model is to reduce redundant action proposals and improve the quality of action proposals.Finally,the above two models are integrated into a Cascaded Boundary Network to generate high quality action proposals,and then more accurate action detection is realized based on these action proposals.Our experiments on THUMOS14 and ActivityNet-1.3 shows that CBN achieve state-of-the-art recall performance.The performance gain is especially remarkable under small Average Number of retrieved proposals(AN),e.g.average recall@AN=50 on THUMOS14 is improved from 37.46% to 43.06%.Further experiments are investigated by introducing proposals generated by CBN to existing action detection framework.CBN still achieves state-of-the-art average map@tIoU on THUMOS14 detection benchmark,e.g.average map@tIoU on THUMOS14 is improved from 45.1% to 48.8%.