Research On Human Activity Analysis Based On Wearable Devices

Human activity recognition(HAR)based on wearable devices has become a hot topic in pattern recognition,computer vision,and multimedia following widespread application of wearable devices(such as smartphones and smart bands)in daily life and the rapid development of deep learning theory.At present,there are many HAR datasets of computer vision and wearable sensors,and many research methods have been proposed and widely used.However,due to the difficulty of collecting and labeling multimodal HAR datasets,there are currently fewer multimodal HAR datasets,leading to the lack of good development in this area.To construct a multimodal HAR dataset,this thesis first studied how to build and a data collection system that can support large-scale human activity information collection,and then studied how to develop a collection plan to ensure the quality and quality of large-scale data collection.Organize volunteers for large-scale collection,and finally proposed MMC-PCL-Activity,a wearable device-based human activity dataset.It contains data recorded from the accelerometers,gyroscopes,heart rates,steps,GPS,weather information,applications usage,and images recorded by third perspective camera for 14 participants performing 16 daily activities.The dataset proposed in this article enriches the diversity of HAR datasets and provides more challenges and research directions for the HAR community.In addition,this thesis also researches HAR based on the MMC-PCL-Activity dataset,including the study of the third perspective image HAR based on the Res Net-50,and the study of the sensor HAR based on the LSTM,CNN,MUL-CNN and CNNLSTM methods,multi-modal HAR of feature fusion and result fusion are studied,and finally a multimodal HAR based on contrastive self-supervised learning is proposed.The experimental results prove the effectiveness of the method.