Research On Salient Object Detection And Visual Servoing Control By Fusing Human Vision

With the development of space and deep sea exploration activities,robots need to carry out various tasks in the extreme environments(such as vacuum,high pressure,etc.).Because of the unstructured characteristics(uncertainty and complexity)in the extreme environments,the robot can not quickly perceive and respond to changes of the scenes.In contrast,the human visual system can focus limited brain processing resources on important objects and prioritize them when facing such complex and uncertain environments.This selective attention mechanism enables human to quickly respond to changes in unstructured environments.Inspired by this biological mechanism,a large number of saliency detection algorithms are proposed to achieve the human visual attention mechanism.The saliency object detection algorithms can achieve rapid positioning of object region in the scene,which is very suitable as the image preprocessing step to achieve quick and accurate object detection,so they have been widely used in many research fields,such as object recognition,object tracking and human-computer interaction.However,most of the researches only focus on the saliency detection of simple scenes,and they are difficult to highlight the saliency object with consistent high and significant values for the scenes with complex texture,volatile background and messy color,.In this dissertation,based on the analysis of the shortcomings of the existing saliency algorithms,two kinds of saliency detection algorithms based on human visual information are proposed,and successfully applied in the robot visual servoing control applications.In this dissertation,the following works are carried out in the aspects of saliency object detection,human eye information modeling,visual servo control and so on.In order to effectively perform saliency object detection with human visual information,a method of human gaze information modeling based on particle filter is proposed.First,based on the temporal and spatial characteristics of eye gaze information,the temporal-spatial model of gaze points is established to describe the gaze point data.Combined particle filter and spatio-temporal model,the gaze point data are processed based on particle filter to generate the density map of gaze information.Experimental results show that the performance of the proposed method is good to express the region of the salient object.Aiming at the problem that the existing algorithms are difficult to deal with complex backgrounds,a saliency detection algorithm based on multi-scale manifold learning is proposed.Using the fixation prediction model and the superpixel extraction method based on the graph to achieve the indication of the salient region and the coarse detection of the salient object.The multi-scale information expression of the scene is realized by the superpixel extraction method.Baed on the proposed manifold learning regularization framework,the saliency estimation of multi-scale scenes is completed.By integrating multiple saliency maps to optimize the coarse detection results,the final saliency object result is achieved.Experimental results show that the F-measure of proposed algorithm is 6.2% more than the second best method on Judd-A dataset.In order to improve the integrity and accuracy of the salient object in the complex scenes,a multi-graph saliency detection algorithm based on human visual information is proposed.By analyzing the characteristics of complex scene,the color and other features are extracted to characterize the superpixel unit.Using multiple features and distance measures construct multiple undirected graphs.The threshold processing is used to threshold the extracted density map of eye gaze points,and the corresponding region is extracted as foreground salient seeds so as to complete the eye gaze information fusion.By analyzing the complementary characteristics of multiple undirected graphs,an improved multi-graph optimization manifold learning framework is used to complete the multi-graph saliency estimation,thus generating the pixel saliency map.Experimental results show that the F-measure of proposed algorithm is 20.7% more than the second best method on Judd-A dataset.On the basis of the saliency object detection,a robot visual servoing control method is proposed to achieve the accurate grasp of the task target.An improved iterative threshold segmentation algorithm is used to achieve the region segmentation of the saliency object,and the image moment feature matching is implemented to achieve the object recognition based on the improved genetic algorithm.According to the configuration of monocular "Eye-in-Hand" camera,the image Jacobian matrix expression of uncalibrated visual servoing is derived,and the visual servoing controller is designed based on Quasi-Newton method.Based on the homography matrix obtained by feature matching,the image feature error is calculated,and the motion control of the robot is completed.Aiming at the problem that the autonomous control of visual servoing is difficult to deal with the unstructured environments,a robot servoing control method based on human vision is proposed.By analyzing the characteristics of human-computer interaction devices such as hand controller and gaze tracking,the corresponding human-computer interactive mapping control methods based on human vision are designed for them to realize the direct control of the robot.Benefitting the idea of collision detection,a method of autonomous control region modeling based on tracking distance is proposed,which can divide the human-computer region into direct control and autonomous control regions.By analyzing the characteristics of visual servo control,a calculation method of autonomous control area radius is proposed,which realizes the automatic switching of control mode.Based on the achievements of the above researches,the prototype system of "spatial non-cooperative target capture" is constructed.The system is described in detail from the system framework and function module development.The saliency detection and visual servoing control methods fused with human vision are verified by the experiments.