A Research Of Monocular Vision-based Intelligently Person Following With An Autonomous Mobile Robot In Outdoor Environment

Since the advent of industrial robots in the later years, being the helper of mankind, the intelligent service robots emergence in the other fields besides the manufacturing industry and walk into our daily life for the first time as a result of the technological progress of the areas of sensor, control, drive and materials. This paper focus on a class of services robot known as the caddy robot which can be an alternative to provide service to golfer, that means the robot be required to have the ability to fully autonomous position the host as target in real-time.Among all commonly used positioning method, vision based position system has unparalleled advantages:first of all the position accuracy of the vision based method is fully capable to meet the need of our system, followed by, the vision sensor is equivalent to the human eye, so it can effectively distinguish between the host and the other persons, and has good scalability, based on the result of mid-level visual processing, a follow-up high-level visual recognition and analysis can be carried out, furthermore, along with the rapid development of related technology and industrial, the cost of a comprehensive visual tracking system has been declining, and is in the affordable range.In order to enhance intelligence and comfort, the vision based mobile robot is required to have the ability to implement real-time person following autonomously and accurately, which means:(1)In start-up step, the previous manual way should be replaced by a friendly man-machine interaction, the whole process of the robot recognized host is performed automatically, and does not need to be restricted in a particular environment; (2) As the core part of the positioning system, we need to seek a simple and effective algorithm for the object tracking in image plane. The vision sensor is mounted on the platform of mobile robot, which brings a number of restrictions, furthermore, outdoor complex environment also increases the difficulty of image processing, all of the above difficulties can be summed up to the high demand of the real-time and robustness of the tracking algorithm; (3) Considering of the system complexity and cost, the system we propose selects to use monocular vision, so how to recover the man-machines distance from the image information is also worth to be studied; (4) there are certain degree of fault tolerance and robustness in the whole process, and the impact of lag caused by image processing can be reduced by the help of control strategy in visual servo step.The major research works and contributions of the thesis include:1.A systematic monocular vision-based intelligently person following with an autonomous mobile robot in outdoor environment is introduced, and complete solution is designed and implement in experiment platform.2.Abandon the old man-made way to manually start the initial step, and an automatic method to start-up the initial was proposed, in order to enhance the intelligence, the threshold in object segmentation step is adaptive, and the system has the ability to distinguish the non-human object, algorithm in this section can serve as a key technology for service robot to recognize the host.3.The mean-shift based algorithm was considered as main tracking algorithm in image plane.To solve the object discontinuous in image plane caused by the motion of the robot, the algorithm which combines the Mean shift and Kalman filter in a novel way is proposed. Based on the description of the relationship between the offset and the motion, the target dynamics with the motion of robot as the external control is consecutively depicted. With the state estimation of Kalman filter as the starting position of the Mean shift and the converge location of the Mean shift as tracking results of current frame, the state estimation is replaced with the converge location of the Mean shift. Two algorithms work alternately, interact with each other, and work well.4.Conventional Mean-Shift based tracking algorithm implements through the single feature space, which brings the confusion caused by the similarity object appearing around the target, thus the mean shift based tracking algorithm through multi-feature space is proposed, when the characteristic in single color space can not provide enough capacity to identify the object, other local features can be used to compensate the discriminate capacity, the paper also proposes the way how to measure the discrimination of features, so the selection of features becomes adaptively.5.Four different distance estimation methods are proposed, for obtaining more robustness, a dominant color descriptor based on multi-color channel was introduced when conducting the splitting of human torso in image plane, furthermore, based on the analysis of contrast tests, integration of different methods was proposed, which means the selection of appropriate method is dependent on the application and condition.6.In visual servo step, a forecast term was introduced to compensate the impact of the lag which caused by the image proceeding, in the other hand, some optimizations were done to improve the overall system performance and fault tolerance, include the introduction of the L2 distance based mean-shift and the proposed'find back' mechanism. The former is to improve the computing speed and the latter is to cope with the situation of loss of tracked target.