Research On Activity Recognition In Ubiquitous Computing Environments By Fusing Multi-modal Information

Effectively utilizing multimodal context information is a promising research direction in ubiquitous computing. However, most existing researches tend to be impractical in ubiquitous environment. These approaches could achieve high performance of activity recognition based on the assumption that sensing devices were fixed on human body and would not slip or rotate during the whole recognition process. Unfortunately, this assumption does not hold for smartphones, which are usually used in an uncontrolled daily living environment. Additionally, most researches attempt to recognize human activity only from accelerometer data or other data (e.g., electrocardiogram). The major flaw embedded in those approaches is that accelerometer data or physiological data only is hardly to achieve reliable performance in real-world applications.To address these problems, an activity recognition method in ubiquitous computing environments by fusing multi-modal information is proposed. Firstly, coordination mapping and shapelets techniques are employed to address the orientation-invariant and placement-invariant issues, respectively. Secondly, a multimodal human activity recognition approach exploiting the diversity of classifier ensemble is presented to utilize multimodal data. Experimental results show that the proposed approach has its superiority on reducing the negative effects of uncontrolled device orientation and placement. The results also show that the proposed approach is able to yield a competitive HAR performance, and has its superiority on exploiting the benefits of multimodal signals.