Research On Automated Planning Methods For Service Task Execution Of Home Robots

With the development and progress of science and technology,home robots gradually enter the home to provide various services for mankind.Service tasks are complex and diverse.When robots perform service tasks,they need to plan the tasks,generate a series of executable action sequences to guide home robots to complete the service tasks.Automated planning method is the basis for home robots to perform various service tasks instead of human beings,and it is the key technology for home robots to realize intelligence.At present,the relevant automated planning method is to generate offline action sequences in a determined environment,which is difficult to adapt to the dynamics and occlusion of objects in the home environment.In addition,home robots lack navigation planning without maps in an unstructured dynamic environment,and lack cognitive manipulation planning that can realize the constraint relationship among tasks,actions and objects.Even for the same service task,the task execution sequence needs to be generated again,which is difficult to realize planning information migration learning.Therefore,when home robots use automated planning technology to generate action sequences and guide robots to perform complex and diverse service tasks in an unstructured and dynamic home environment,they still face many challenges such as the effectiveness of task planning,the autonomy of navigation planning and the constraint of manipulation planning.The lack of independent planning ability for service task execution has become a key bottleneck restricting the development of home robots.How to make the home robot have the ability of automated planning applied to the execution of service tasks,so as to ensure the efficient and reliable automated execution of tasks in the home environment,is the key to promote the home robot to enter the home and realize intelligent service.This thesis takes the automated planning for service task execution of home robots as the starting point,and carries out systematic research from task planning,visual semantic navigation planning without maps,cognitive manipulation planning of objects and Knowledge engine.The main research results are as follows:(1)In order to enable home robots to obtain sufficient environmental information and plan tasks to produce effective offline action sequences when facing the uncertainty and incompleteness of home environmental information,an offline task planning method based on semantic knowledge and probabilistic inference is studied.In order to effectively obtain home environment information,the construction method of semantic knowledge model of home environment and the probabilistic inference strategy of object location were studied,and the object location information was obtained by knowledge model and probabilistic inference.On this basis,an improved hierarchical task network planning method is proposed to generate offline action sequences,and a task replanning method is proposed for task execution failure.The experimental results show that the semantic knowledge model and probabilistic inference can make the robot effectively obtain the home environment information,reduce the search space during robot task execution,improve the efficiency of task execution and enhance the effectiveness of task planning.(2)Aiming at the problem that it is difficult to generate online action sequences when objects are blocked in the home environment,a hybrid offline and online task planning method based on static object-level semantic map and probabilistic inference is studied.Firstly,the construction method of static object-level semantic map is studied to obtain the location information of static objects in the home environment,and the location information of dynamic objects is obtained through probabilistic inference.Secondly,the automatic domain composition method of probabilistic programming domain definition language is proposed,and the object occlusion observation model is introduced into the partially observable Markov decision-making process.On this basis,a hybrid offline and online task planning method is constructed,and the dynamic switching of the two kinds of sequences is realized through the designed planning switching mechanism.Experimental results show that this method can effectively solve the task planning problem under the condition of object occlusion,and enhance the robustness and intelligence of task execution.(3)Aiming at the problem of robot automated navigation planning without map in an unstructured and dynamic home environment,a visual semantic navigation planning method based on semantic priori and visual transformer is studied.Firstly,the prior knowledge graph and the current knowledge graph obtained in real time are established to highlight the semantic location relationship between objects.The current knowledge can update the prior knowledge graph regularly.and the semantic prior information is combined with target information as semantic feature information.Secondly,in order to improve the long-time navigation ability and generalization ability of the robot to the unknown environment,the environment observation information is extracted and combined with the semantic feature information to form the storage and update mechanism of visual features and semantic features.Then,the temporal and spatial dependencies of visual features and semantic features are extracted through visual transformer,which is used as the input of the navigation strategy model composed of long short-term memory network and multi-layer perception.Experimental results show that this method can improve the generalization effect of robot navigation planning in an unstructured and dynamic home environment.(4)In order to make the robot understand the constraint relationship among task,action and object,and select the appropriate action to manipulate the object according to different tasks during task execution,a robot cognitive manipulation planning method based on object affordance and logical reasoning is studied.Firstly,in order to improve the accuracy of object affordance segmentation,a convolution neural network based on attention mechanism is constructed.Secondly,aiming at the diversity of service tasks and objects in the home environment,the object ontology and task ontology are constructed to realize the knowledge representation of objects and tasks respectively,and the object-task affordance is established through the casual probability logic.On this basis,the robot cognitive manipulation planning model is designed by using Dempster-Shafer theory.The model can infer the most suitable task-oriented object manipulation position.Experimental results show that this method can effectively improve the cognitive manipulation ability of the robot and make the robot complete manipulation tasks more intelligently.(5)In order to enhance the learning and migration ability of automated planning for robots,cloud based home robot automated planning knowledge engine is studied.Knowledge engine is composed of online knowledge base and cloud engine,which can effectively combine task planning,visual semantic navigation planning,cognitive manipulation planning and visual perception methods.The Online knowledge base can store the generated planning information and visual detection information,while the cloud engine can deploy planning methods and visual detection methods,and provide retrieval and storage tools for task execution information.In addition,the affordance instance segmentation network and the affordance based pose estimation method are proposed,which are combined in the visual detection module of the knowledge engine for object affordance detection and pose estimation respectively.Experimental results show that the cloud based knowledge engine can realize the effectiveness of robot automated planning and enhance the learning and migration ability of planning methods.