Research On Multi-modal Activity Recognition On Singal Processing Technology

With the continuous development of sensor technology,more and more types of data are applied to the research of activity recognition.A single type of data can only give a one-sided description of the current activity of the target,while adding different types of data can add multi-dimensional description supplement to the activity of target.Therefore,in order to get a higher recognition accuracy,multi-modal activity recognition technology using or integrating multi-source data features has gradually developed into a new research topic.In this paper,we use three kinds of data sources: EEG signal,image sequences and motion signals,which are composed of multi-modal system to study activity recognition.In view of the three basic steps of signal acquisition and preprocessing,feature extraction and modeling recognition in traditional behavior recognition research,this paper carried out the following work:(1)In the signal preprocessing step,the traditional nonlinear time-frequency analysis denoising method has poor effect on the special peak noise in the EEG signal.In this paper,an iterative algorithm combining singular spectrum analysis and low-rank decomposition method is proposed to suppress the peak noise in the EEG signal,making the output EEG signal more reliable.The experimental comparison shows that the proposed method is superior to the current denoising methods.(2)In feature extraction step,EEG signals are decomposed into components of different frequency bands by conventional filtering method.This method will introduce nonlinear phase distortion to these components.Therefore,in this paper,the EEG signal is decomposed into several intrinsic mode functions by using the empirical mode decomposition method,and these intrinsic mode functions are further divided into four groups according to certain rules.Finally,several physical features,such as energy and entropy,are extracted from the EEG signals of different frequency bands synthesized by each set of intrinsic mode functions.In addition,the corresponding features of image sequence and motion signal are extracted to obtain different types of feature vectors.(3)In modeling step,this paper combines the four multi-classification methods of SVM,random forest,XGBoost and Light GBM with different feature vectors,and models single algorithm for individual,single algorithm and integrated algorithm for multi-person respectively.The final experimental results show that multi-source signals can achieve better recognition effect than single source signals in activity recognition research.For the feature extraction method of EEG signal,the new method is superior to the conventional filtering method.The results based on the integrated model are superior to the single algorithm model,and the optimal classification accuracy of both the single algorithm model and the integrated model reaches more than 90%.