The Research And Application Of Zero-shot Learning In Health Event Recognition

The development of technologies such as the Internet,the Internet of Things,and cloud computing has led to individual-centric health big data.The self-centric data recorded by smart phones and various wearable devices continuously and objectively records the individual's lifestyle,which can be seen as the external expression of the individual's health.By recognizing these health events,effective quantification and real-time monitoring of individual lifestyles can be achieved,which helps to analyze individual health condition,estimate disease risk,and prevent disease.Health event recognition can be regarded as the category of egocentric activity recognition in human activity recognition.With the development of deep learning,breakthroughs have been made in human activity recognition methods based on supervised learning.However,this method often requires a large number of labeled samples,which is labor-intensive and difficult to expand to few-shot or even zero-shot,and the model generalization ability is very weak.Zero-shot learning can use the correlation between known and unknown categories to transfer knowledge in data from known categories for identification of unknown categories,which provides innovative ideas for solving the above problems.At present,most of the zero-shot learning methods are to study the object recognition problem of static pictures.Applying such methods directly to video sequences will cause the problem of missing timing information.Recently,some work has begun to explore the use of information such as timing and optical flow,but still ignoring the scene information that is essential for activity recognition,the scene often directly determines whether an activity can occur.In addition,the existing zero-shot learning methods are mostly focused on the visual field,without combining information of other modalities.In response to these problems,in the context of health event recognition,this paper studies how to effectively use scene information to improve the performance of zero-shot health event recognition,and on this basis,how to conduct multi-modal health event recognition.The main work of this article is as follows:(1)A zero-shot health event recognition method incorporating scene information is proposed.This method fully considers the important role of scene information for the recognition of health events,uses objects and scene information to jointly establish a visual space for video instances,and explicitly models the relationship between health events,objects and scenes through a two-stream graph convolution network.The addition of scene information not only makes the attribute characteristics of the video instance in the instance branch more robust,but also promotes the information transfer between the seen and unseen classes in the classifier branch,improves the ability to recognize unseen categories.(2)A multi-modal zero-shot health event recognition method that integrates triaxial acceleration information is proposed.The method combines triaxial acceleration and video with multimodal fusion according to the semantic correlation and information complementarity between different modal data,and further,through a fusion module to abstract into a more discriminative fusion feature,which makes the generated classifier of the classifier branch is more accurate.The two branches jointly learn and optimize together,which improves the accuracy of zero-shot health event recognition.