Towards Deep Understanding Of Human Activity In Wearable Computing

Wearable devices,including smartphones,smart watches,smart glasses,etc.have become an indispensable part of people's daily life.Recent years have witnessed the considerable development of devices' computing capacity and sensing ability.Besides making phone calls,smart wearables are providing us with a lot of advance functions by sensing the context of the users.However,faced with rapid growth of the user population and the volume of user data,the ability of understanding users' needs to be improved urgently.Subscribers tend to wear smart devices while working,eating,playing and even sleeping,while off-the-shelf applications can only deal with limited activities,such as walking,shaking,etc.In this work,we manage to empower smart devices with the ability to understand people and their activities.Thus we can achieve more efficient management over user data and better user experience.To this end,we carefully study people's activities and the corresponding data collected by the embedded sensors(e.g.orientation,location,acceleration,angular velocity,etc.).Firstly,we focus on the general movement of users and propose a model to describe video data,which take up most of a user's storage.The adoption of Field of view(FoV),i.e.the content-free feature,helps to achieve efficient video segmentation and management.The proposed content-free video similarity model shows a high consistency with contentbased approach.For both client side and server side,content-free features lead to much smaller memory cost and the computation overhead is greatly reduced.Meanwhile,by uploading content-free features,negligible cellular traffic is needed and the privacy threat can also be avoided.Secondly,we take the first step towards the hierarchical semanteme of people's activities,and realize the deep understanding of activities by the proposed multi-level model.Based on the analysis of the semanteme model,we propose the brand new concept of activity search,by which people can upload an activity and retrieve all the corresponding data segments in the database.The comprehensive experiment shows that the proposed semanteme model can automatically extract the descriptive information of people's activities,and the extracted hierarchical semantic feature shows a resistibility to the interference caused by different people,habits and timescales.Besides,our model is fully unsupervised so that we can avoid the labor cost of annotating data.Last but not the least,we further pay our attention to the more subtle activity of handwriting,and propose an ubiquitous input scheme using only Commercial-off-theshelf(COTS)devices.The proposed scheme can adapt to user's different postures and automatically build a virtual whiteborad.What's more,we propose to decompose the activity of writing an English word into writing a sequence of strokes.The proposed scheme can pinpoint each of the stroke and the word recover model can learn the mapping between each word and its stroke sequence.As we can enumerate all the possible strokes in English words,such mappings can be automatically generated so the arduous overhead on collecting training data can be greatly reduced.