Research On Human Complex Activity Recognition Baesd On Multi-Sensor Data Fusion

With the development of science and technology and the popularization of sensor technology,sensor-based human activity recognition has been widely used in various fields,including intelligent medicine,smart home,sports activities and so on.In the past decade,human motion recognition technology based on wearable inertial sensors has achieved great success [2].However,most of the current studies only use single sensor data to identify several specific human activities.However,in practical applications,most of the activities in real life are concurrent and complex activities,such as sitting to eat,standing to wash dishes and so on.These activities are related,have context information,and these complex activities occur less frequently than basic activities such as sitting and walking,short time,high similarity,poor discrimination and so on.At present,the recognition accuracy of complex activities is low,and the recognition of complex activities by a single sensor has limitations,which can not achieve relatively accurate recognition results.In recent years,wearable devices and smart phones are equipped with various sensors,such as accelerometers,gyroscopes,magnetometers,pressure gauges and wearable cameras,which can be used for activity recognition.However,it is a challenging problem to combine data from multiple heterogeneous sensors for complex delivery identification,which requires innovative research solutions.Therefore,we are research the complex activity recognition method based on multi-sensor data in this paper.Firstly,the data fusion strategy based on multi-sensor is studied,and the advantages and disadvantages of three-level data fusion strategy based on traditional machine learning are studied and analyzed.Further introducing deep learning method,because convolution neural network has the function of automatic feature extraction and hierarchical overlay,according to the idea of three data fusion strategies,combining with deep convolution neural network,three data fusion methods based on convolution neural network are obtained.In order to verify the effectiveness of deep learning,the accuracy of activity recognition is compared with three data fusion strategies based on machine learning.The experimental comparison shows that the accuracy of activity recognition based on deep learning is 20% higher than that based on machine learning,and the average accuracy of activity recognition based on deep learning is 85%.However,the data fusion method based on deep convolution neural network only has a low recognition accuracy for individual activities,especially for rare and confusing activities.In order to solve the above problems,a deep hybrid neural network model of CNN + GRU is obtained by combining the data fusion model based on convolution neural network with the cyclic neural network.The average accuracy of the model is 90% and that of the non-periodic activity is 9%.Aiming at the recognition of concurrent complex activities in daily life,combined with the CNN + GRU hybrid deep learning model based on multi-sensor data fusion,multi-task learning is introduced to recognize concurrent complex activities.The concurrent complex activities are divided into several sub-tasks,each of which is a kind of atomic activity.By using the joint training method of multi-task learning,we can share the structure of in-depth learning network,learn the correlation between various activities,promote each other's learning,and improve the recognition accuracy of easily confused activities.Experiments show that the average accuracy of 21 concurrent complex activities is 95% by using multitask learning,which improves the generalization performance of complex activities recognition.In this paper,an open data set is used to validate the experiment.The data set is collected entirely from real-life activities.It is collected by seven different sensors on smartphones.It includes many activities with context information.In this paper,the data set is analyzed and preprocessed,which solves the problems of data missing and data imbalance in the data set.Further,the complex activity recognition model based on in-depth learning proposed in this paper is deployed on smart phones,and an application of complex activity recognition is developed on smart phones to verify the effect of the model on complex activity recognition in real applications.