Study Of Human-robot Interaction Technology Based On Vision

With the development of the society, social robots enter into people’s daily lives gradually, and they can cause users’ social behavior to provide services by close-interaction with the users in daily environment. Human-Robot Interaction(HRI) is the main ability for the social robot. The interaction ability means that the robots not only have a wealth of awareness and responsiveness, also can make an appropriate interaction in different occasions. At present, the research and development of social robot is often limited in specific application field by specialized technical personnel, and the robot’s intelligence ability is not good enough to adapt to different situations independent, both of which influence the spread of the robot. This thesis focuses on HRI of social robots that provide services in public place, and does research in two aspects to improve the HRI ability of social robot. Firstly, study the human detection, tracking and interaction state recognition to improve the perception of the social robot; secondly, study the method by which user can update the robot’s interaction function through updating interaction knowledge. By the above study, user can develop the robot’s function according to the need of application by update interaction knowledge based on the robot platform that has rich perception and responsiveness. The main contributions can be summarized as bellows:(1) In order to improve the performance of foreground detection in HRI, a new background subtraction method based on image parameters is proposed, which helps to improve the robustness of the existing background subtraction method. The proposed method evaluates the input image and foreground results according to the image parameters representing the change features of the image, such as the color, luminance and the edge character. It filters the image that may break the background model; detectes the broken probability of background model and rebuilds the background model when the model is broken. The above methods can make sure that the background subtraction method detected foreground object with correct background model. Experimental results of typical interaction scenes validate that the proposed method helps to reduce the broken probability of background model and improve the robustness of background subtraction.(2) A tracking algorithm for HRI is proposed as a solution to the problem of interference by similar human object when a robot is tracking specified human object in HRI. By the screening of candidate targets from the interference regions and particles distribution model based on overlap-rate, particles can be converged to all candidate targets, and be reduced. The summation of weighted-distance-error and the target size are regarded as cluster condition to divide particles into corresponding particles of the candidate targets. The best candidate target is selected as the tracking result. Experimental results show that this method can avoid the interferences of surround similar objects and tracking target accurately, and has robustness and real-time property.(3) To solve the problem of multiple similar objects and camera motion in target tracking, a target location and tracking method based on local background feature points is proposed. Firstly, predict target position by the position relation of the matched feature point s and target. The matched feature points surround target in previous image. Then search candidate objects start from the predicted position by combining particle filter and mean shift. Finally, the similarity of each candidate object is weighted by the distance between candidate object position and predicted position. The final target is the candidate object with the highest similarity. Experimental results show that the proposed method performance well under the situation of multiple similar objects and camera motion, and the location method has real-time property and can combine other tracking method to improve the tracking effect.(4) In order to let robot recognize the change of interaction state and react proactively, an interaction state recognition method based on Bayesian Network is proposed. Firstly, the algorithm of interaction state recognition is gained by constructing interaction migration model and defining migration condition. Secondly, the interaction trend Bayesian model based on face direction and its motion information is constructed. After calculating the dynamic state probability of face direction node and motion node, the state probability of interaction tread is got by Bayesian network inference. At last, the algorithm of getting face feature and motion feature are studied. The study of interaction state recognition is the base of perception application of the HRI software framework.(5) A HRI software framework for user updating robot interaction function by editing interaction knowledge is proposed. Firstly, a HRI software architecture based on abstract environment is proposed, the structure of architecture and mechanism of abstract environment are studied. Secondly, the scenario file based interaction knowledge and its interpreter both are designed, and the development platform for user updating interaction knowledge is developed. At last, the robot platform is developed. Tests for user updating interaction functions and interaction state recognition are conducted. Experimental results show that the results of interaction state recognition are conformed to the actual situation, and user can update robot’s interaction function by editing interaction knowledge.