Research On Dynamic Scene Understanding Based Learning From Demonstrations For Robots

As an interdisciplinary field,with the development of computer,machinery,electronics and other technologies,the research results of robot have been widely used in manufacturing,medical rehabilitation,underwater operations,aerospace,military and many other fields.In the interaction and cooperation with robots,people increasingly hope that robots can master all kinds of operation skills like humans,and use these skills freely to help people complete all kinds of tasks in production and life.This can not only liberate people from some simple,repetitive and dangerous work,but also provide more direct and rich human-computer interaction mode.However,Such kind of robot is often a typical highly nonlinear and strong coupling dynamic system.When it completes various tasks,its nonlinear dynamic characteristics are very significant,and the actual existence of various complex uncertainties and disturbances also seriously affect the control quality of the robot system.Therefore,the control of the robot often needs professionals to design controllers to adapt to Different tasks.This will make it difficult for non professionals to control the robot.In addition,the robot programming and debugging often need to use a variety of sophisticated experimental equipment,which makes it difficult to use common equipment to control the robot to perform tasks in general civil environment.Therefore,this paper hopes to build a framework,which can make the robot understand the dynamic scene with a simple visual device,and based on the understanding of the dynamic scene,learn the human control strategy through teaching,so as to realize the intelligent control of the robot finally.The research content of this project mainly includes three aspects:1.Understanding of dynamic scene.The robot can capture the semantic information and spatial information of the dynamic scene through a commonly used RGB camera.This determines that the robot can analyze which pixel area different objects or backgrounds belong to only through the two-dimensional image information;at the same time,the depth information corresponding to the pixel can be reconstructed through the two-dimensional image information to obtain the spatial stereo information of the processed scene.Through the synthesis of semantic and depth information,it is easy to get the moving track of the object in space through video stream.Thus,the robot can complete the teaching process through non-contact demonstration or video,which ensures the convenience and safety of teaching.2.Teaching and learning control of mechanical arm.The 3D trajectory samples transformed by video stream can be used to teach robot trajectory learning.Teaching and learning makes robots more flexible and adaptable in unstructured environment,so it is widely used in robot task programming.The generalization ability of teaching learning is more advantageous than the traditional programming method.However,in the point-to-point teaching learning trajectory,there is often a contradiction between stability and Trajectory Accuracy,which is particularly obvious in the learning of complex trajectory.In order to solve this contradiction,so that the robot can learn the precise control strategy,but also to ensure the convergence to the target point,we need to improve the original teaching learning algorithm framework,so that it can achieve this goal.3.Teaching and learning control of mobile balance vehicle.The control of manipulator has relatively high degree of freedom,so it is easy to teach by vision.In order to make the robot understand the dynamic scene and make the teaching framework more general,it is necessary to study the teaching and learning problems of more constrained robots.The mobile balance vehicle itself has non integrity constraints and under drive constraints,so it is necessary to establish a teaching and learning framework with constraints,so that the robot can learn the strategy of human through visual teaching under the condition of satisfying constraints.In this case,the trajectory of the mobile balance vehicle also needs to overcome the contradiction between accuracy and stability as much as possible,that is,to learn human control strategies as much as possible to ensure convergence to the end of the target.