A New Approach For Real-Time Action Tracking And Recognition With Lightweight Architecture

In recent years,a lot of research about action recognition are published.Nevertheless,there are only a few researches that focus in real-time applications,which normally include a human tracking system and can work at fast speed.The existing works still have some limitations that are unstable detection in the human tracking part and slow prediction.To solve these problems,this paper proposes lightweight deep learning architecture and a new combination of modality that suitable for real-time action recognition with a tracking system.The first step is to detect humans in each frame by Tiny-Yolo and to track the person by Euclidean distance from their centroid.The second is to transform each input frame into a new modality,MRGB-Diff.The last step is the action recognition part,we modify Conv3D and Conv-LSTM to be a lightweight version for comparing the performance.We use 2 datasets,KARD and BUPT-5daily,to compare the performance of the approach.We got 79.05%and 85.13%accuracy on BUPT-5daily and KARD,respectively.For speed,Conv-LSTM and Conv3D can achieve 25.4 ms and 14.9 ms,respectively.The result of the experiment shows that our approach can work efficiently in terms of accuracy and speed in real-time both.