Human Action Recognition Based On Weakly Annotated Sensor Data

With the popularity and development of the wearable devices such as smartphones,human activity recognition(HAR)based on sensors has become a key research area in human computer interaction,and many applications benefit from HAR include sports activity detection,smart homes,and health support,et al.In recent years,with the rise of artificial intelligence and deep learning technology,more and more researchers have begun to use deep neural networks to solve HAR problems,and many HAR approaches and theories based on deep learning have been proposed.However,most of these deep learning methods belong to the category of supervised learning,which requires massive strictly labeled data.In comparison with image and video data,activity data recorded from the inertial sensor can not be labeled by human annotation,which makes the process of sensor data collection and labeling very labor-intensive.For these contents,the main work of this article is as follows:1.The concept “Weakly Labeled Sensor Dataset” has been proposed,in which the samples include not only the labeled activity but also the background activity and noisy data.Compared with the strictly labeled data,the weakly labeled data does not need strictly labeling,which can facilitate the process of sensor data annotation and makes data collection easier.2.The Attention-based Convolutional Neural Network(ACNN)has been proposed to solve weakly labeled activity recognition task.The model can compute the compatibility between the global features extracted at the final fully-connected layers and local features extracted at a given convolutional layer,which can amplify the salient activity information and suppress the irrelevant and potentially confusing information by weighing up their compatibility.3.ACNN can only handle the weakly labeled dataset whose sample includes one target activity,as a result,it limits efficiency and practicality.So,a recurrent attention network(RAN)was proposed to solve the weakly labeled multi-activity recognition task.The model which uses a recurrent neural network can repeatedly perform steps of attention on multiple activities of one sample and each step is corresponding to the currently focused activity.This paper evaluates the classification performance of the ACNN and RAN models on the human activity datasets such as UCI HAR,OPPORTUNITY,UniMiB-SHAR,WL,and SWL respectively.It is proved that ACNN and ran models outperform traditional convolutional neural networks in single category and multi-category human activity recognition tasks with weakly labeled sensors.Meanwhile,by analyzing and processing the attention heat map generated by the attention mechanism in ACNN and RAN model,a method of converting compatibility score into compatibility density is proposed to realize the target action localization task in a weak labeled sensor data segment.Experiments verify that the method can achieve effective action location.