Human Action Recognition And Understanding In Intelligent Space

Nowadays, the medical care and accompaniment of solitary senior citizens has become a special social problem due to the aging society tendency in many countries. One accredited strategy to deal with this crisis is to develop intelligent service robots. Particularly, human behavior recognition and understanding are two essential prerequisites for service robots to function correctly. At present, the research of behavior recognition and understanding based computer vision is still in its preliminary stage. It is an extremely challenging task for the robots to adapt to the variety of human behavior patterns and the complexity of home environment. In other words, it is very difficult to accomplish such complicated mission as providing correct service based on the recognition and understanding of human behavior as far as the currently robot development level is concerned. Therefore, in order to solve this kind of problems, a vision-based method is presented in this thesis that can achieve human behavior recognition and understanding combined with the intelligent space technology. This method makes it possible to accompany and take care of solitary elderly based on the intelligent service robots and the intelligent space technology. It can be shown that the study of the intelligent space and behavior recognition and understanding in intelligent space have important theoretical and practical meanings.In this thesis, five essential aspects concerning behavior recognition and understanding in intelligent space are studied which include the research and construction of ISUS (Intelligent Space for Understanding and Service), detection and tracking of moving object, typical action recognition, abnormal action recognition, and behavior recognition and understanding. Firstly, the construction of ISUS is studied. ISUS is not only a physical space of human activities, but also an intelligent space which can perceive human's behavior and provide assistance to human in it. The visual and other relevant information of ISUS is required in the course of study on the behavior recognition and understanding. Therefore, ISUS is the foundation of the following research. Moreover, object detection and tracking in the background of the old solitary people in home environment is studied. The accurate target detection and real-time tracking are two prerequisites for feature extraction in the action recognition which will be shown in the following. Based on the correct detection and tracking of objects, the recognition of common daily actions and abnormal actions can be achieved. The correct recognition of the daily actions and abnormal movements are foundations of the behavior perception and intention recognition. Lastly, behavior perception and intention recognition in ISUS is researched, and the intention recognition of the normal behavior and the abnormal behavior recognition are achieved. Based on the above five aspects, ISUS can provide the corresponding service to human directly or indirectly according to the recognition result. The main research contents and results are shown as follows:(1) In view of the inefficiency of iSpace, the concept of ISUS and the idea of execution distribution are discussed, which are used to effectively improve the service efficiency and reduce the burden of the robot. The layered architecture of ISUS is designed from the perspective of intelligent space standardization, which makes the system more compact, clear and convenient to be built. Meanwhile, this method is convenient for task allocation. Moreover, the constructions of ISUS, task planning and information flows composition in ISUS are studied based on the above researches.(2) The GMM algorithm is improved according to the environment characteristic of ISUS. It improves the capability of models to timely reflect the background changes. Therefore the effect of target segmentation is increased, the mean shift algorithm is improved and a template and target scale updating strategies for the ISUS environment is proposed. The robustness of object tracking is raised combined with a Kalman filter. In this paper, object locationing and tracking technology combined wireless sensor networks and computer vision is adopted. Many problems, such as the heavy occlusion and re-capturing of objects during object tracking, the difficulty of target hand off and object matching in multi-camera object tracking, are solved. Object relay tracking in multi-camera system is also realized with this method. (3) Human action recognition is the foundation of behavior perception and intention recognition, and the feature selection is a key to it. Good features can not only be conducive to recognition, but also reduce the amount of calculations. In view of the problem that it is difficult to improve the recognition accuracy using the common features, this thesis presents a methodology for typical action recognition using the contours of human posture, which effectively improves the action recognition accuracy. In order to overcome the shortcomings of human body contour apt to be effected by segmentation results, the silhouette of human action is used as recognition feathers after dimension reduction which improves recognition reliability.(4) The abnormal action recognition in home environment is studied which includes the falling, abnormal gait and abnormal walking status. Especially, as it is difficult to distinguish falling from lying down using existing methods, a new algorithm is proposed in falling recognition, which increases the accuracy of falling recognition using the static and dynamic features and human location in ISUS and physical condition. Moreover, different from the gait-based human identification, the purpose of abnormal gait recognition in this thesis is to infer whether an abnormal situation has occurred. Therefore, gait parameters reflecting stride changes are defined by the human gait characteristics and the abnormal gait recognition that can be achieved by analyzing these parameters. Lastly, the abnormal walking state recognition is achieved by judging human walking trajectory.(5) Behavior perception and intention recognition belong to the high-level processing stage. In this thesis, the human locationing in ISUS is researched first. A new algorithm of human locationing in ISUS is proposed using height models combined with posture recognition, which reduces the effects of body posture changes to locationing accuracy. The locationing accuracy is further improved using the combination of multi-camera locationing results by the defined position confidence coefficient. The normal behavior perception and abnormal behavior recognition are researched based on human location. In normal behavior perception and intention recognition, the concept of key points in ISUS is discussed, and key points and area of ISUS are defined. Then a new behavior perception algorithm based on place-driven is proposed, which achieves human's intention inference through interrelating key points and its attribute. In abnormal behavior recognition, a key point based method of abnormal event recognition using activity route is introduced. Then the concept of key point's duration histogram is put forward and used to recognize abnormal behaviors. Finally, the abnormal degree of a behavior is obtained by the defined evaluation index of habit changes.