Research On Robot Visual Perception And Motion Planning For Human-Robot Collaboration

With the development of social economy,people's demand for customized products is becoming more and more vigorous.For the demand-oriented manufacturing industry,the most significant change is the rapid growth of demand for multi varieties and small batches of products.So,the production mode of industrial manufacturing also changes.As the core equipment of intelligent manufacturing,industrial robots need to satisfy the requirements of flexible customization.Thus,human-robot collaboration system with flexibility and intelligence has become an important development direction of technology application of industrial robot.For the human-robot collaboration system,there are complex and dynamic unstructured obstacles,which need the robot to be able to perceive the environment and target information and make relevant decisions according to the task requirements for performing the task quickly,safely and controllably.Among them,the technologies of visual perception and motion planning are the key to the robot's flexibility and intelligence.Therefore,aiming at the problem of vision perception and motion planning in unstructured environment such as human-robot collaboration,the main research contents of this paper are as follows:Firstly,in response to the problem that two-dimensional image information such as simple textures and colors is difficult to satisfy the requirements of intelligent and flexible robots in an unstructured industrial environment with diverse varieties and unordered materials,this paper proposes a method to estimate the pose of the target object by combining semantic segmentation and point cloud registration based on the RGB-D information obtained by the vision sensor.Aiming at the problem of inaccurate segmentation results of the full convolutional neural network model,a convolutional layer with holes was introduced to improve the full convolutional neural network and an improved network model was trained on a homemade data set based on transfer learning technology.Then,the modified full convolutional neural network is used to perform semantic segmentation on the color map.And the point cloud data of the target workpiece is extracted with depth information.By registering with the initial point cloud,the relative pose of the current workpiece is evaluated and target pose information is provided for subsequent motion planning.Finally,experimental results prove the practicability of the method of pose estimation.Secondly,the autonomous motion planning of the robot guided by the pose estimation information of the target object is an important manifestation of robot intelligence.Therefore,in order to solve the problem of motion planning in a static and unstructured environment in the process of human-robot collaboration,a motion planning algorithm of heuristic-oriented Rapidly-exploring Random Tree is proposed.Aiming at the problems of slow speed of convergence and high cost on path of the classical algorithm of Rapidly-exploring Random Tree,firstly,a target-oriented probability threshold is introduced in the stage of generating random sampling point to increase its probability of expanding to the target node and accelerate the algorithm's convergence speed.Then,in the new generation stage of paths of the exploration tree,the sampling points and paths that optimize the motion cost are selected based on the heuristic graph search algorithm.Experimental results of simulation show that the improved algorithm has significant effects.Finally,an experimental platform of robotic intelligent sorting for cluttered workpiece scenarios is established to verify the practicability of the improved algorithm in actual unstructured work scenarios.Thirdly,for solving the problem of safety of human-robot collaboration in a dynamic unstructured environment,a multi-visual perception-based robot online trajectory planning method is studied.First,an environment modeling and updating method based on information fusion of multiple depth cameras is proposed.A three-dimensional raster mapping model of multi-camera depth images and robot workspace is established offline Combined with the real-time environment information in the online phase,the occupation status of the three-dimensional grid in the robot's workspace is quickly determined,and the minimum distance between the robot and the obstacle is evaluated in real time.Then,based on the reactive obstacle avoidance strategy,the potential field force of the artificial potential field method is improved.And it is transformed into the robot joint speed.The robot is controlled to execute the obstacle avoidance trajectory from the speed level.At the same time,in order to take into account the dynamic obstacle avoidance response speed and work efficiency,an obstacle avoidance trajectory adjustment strategy based on the relative position and speed of obstacles is proposed.Finally,the verification platform of a collaborative robot and dual global depth cameras is designed and built.And real-time obstacle avoidance trajectory planning experiments are conducted to verify the effectiveness of the dynamic obstacle avoidance method.Fourthly,aiming at the problems of high cost of path of reactive local obstacle avoidance algorithms and poor real-time performance of existing global motion planning methods,a replanning algorithm of path is proposed,which satisfies the requirements of completeness and real-time performance.First,the dynamic road map method was improved based on the motion primitive method and the hierarchical structure method The offline map was used to establish a road map of collision mapping between the robot pose in the configuration space and the three-dimensional grid in the work space.Then,based on the environment modeling method of information fusion of multiple depth cameras,the occupation status of the three-dimensional grid and the corresponding offline collision mapping road map are updated online.And combined with the heuristic graph search algorithm,the motion path is re-planned in the updated collision mapping roadmap.Finally,the validity of the motion planning method is verified by experiments.Finally,in response to the needs of unstructured industrial for intelligent and flexible robotic vision perception and motion planning,a comprehensive simulation experiment of human-robot collaboration was designed to glue workpieces and install seals.First,the operation procedures of human and robot were planed according to the operation content and process requirements.Then,the pose estimation experiment for intelligent sorting tasks of chaotic workpieces and the robot motion planning experiment in static/dynamic unstructured environment for human-robot collaboration are carried out in this order.Finally,the experimental results verify the accuracy of pose estimation and the effectiveness of the motion planning algorithm.To sum up,for the problem of vision perception and motion planning in the process of human-robot collaboration,firstly,the method of pose estimation for the working object in the unstructured environment formed by the disorderly stacking of various kinds of workpieces is studied.On this basis,the method of environment perception,modeling and robot motion planning for the scene of human-robot collaboration is further studied to realize the safe movement of robot in the static/dynamic unstructured environment.This has an important role and significance for improving the level of human-robot collaboration,and then improving the production efficiency and production quality of robots.