Research On Service Task-oriented Human-like Efficient Object Search Mechanism For Household Robots

In the household environment,the service tasks performed by the robots often involve one or more objects.The efficient search of target objects is an important foundation and key component for the successful completion of service tasks.At present,the frontier research on object search has achieved the search of single or multiple specified target objects by the robot,but has not designed the corresponding object search method for service-oriented tasks,and has ignored the influence of the order and execution characteristics of service tasks on the object search strategy and results,which makes the robot inefficient and less intelligent during performing service tasks on object search,affecting the service quality and execution efficiency of the robot,and limiting the further development and application of service robots.To this end,aiming at the problem of robot object search in the context of service tasks,this dissertation studies a service task-oriented human-like efficient object search mechanism for household robots systematically by simulating human strategies of guiding object search through service task cognition,and obtains the following research results:(1)In order to enable the robot to have the ability of service task cognition,a multi-domain knowledge modeling method for service task cognition is studied in this dissertation,which provides powerful knowledge support for robot object se arch planning under the background of service tasks.Firstly,the multi-domain knowledge framework containing service knowledge,scene knowledge and object knowledge is constructed according to the knowledge requirements of service task cognition.Secondly,an autonomous generation method of the multi-domain knowledge based on data mining and knowledge extraction is designed to reduce the cost of knowledge modeling and improve the efficiency of knowledge generation.Then,this dissertation proposes a variety of service task attributes to describe the order and execution characteristics of the service tasks,designs the methods of knowledge fusion,knowledge correction,knowledge inference,and so on,and builds a hierarchical and associative ontology knowledge model of "service task-scene-object" to enhance the cognitive ability of the robot for service tasks.The experimental results show that the constructed multi-domain knowledge model based on the above method has fast response time and strong knowledge reasoning ability,which can improve the cognition ability of robot on service tasks,and provide powerful knowledge support for the robot object search planning in the context of service tasks.(2)In order to simulate the object search planning strategy of human beings,this dissertation constructs an object search planning method based on the multi-domain knowledge,which realizes efficient and intelligent object search planning for the robot in the context of service tasks.Firstly,service task cognition is performed based on the constructed multi-domain knowledge,and the faced object search problems are classified to process.Under the support of the service task attributes,different planning methods are designed to generate search sequences including target family scenes and target objects to guide the robot to search for objects.Secondly,aiming at the problem of object search planning in the context of single service task,an object search planning method based on the object number attribute and object order attribute is designed to combine object knowledge and scene knowledge to generate an object search sequence.Moreover,aiming at the challenge of object search planning for multiple service tasks,an object search planning method based on the step clearance attribute is designed so that the robot can complete the object search task efficiently in an interspersed way.The proposed method is tested in both simulation and real-world environments.The experimental results demonstrate that the proposed method improves the efficiency and intelligence of the robot in search for objects in the context of service tasks.(3)Aiming at the problems of the low accuracy of family scene recognition due to the limited observation field of the robot and the poor adaptability of family scene recognition model,a wide-field adaptive family scene recognition method based on multi-type cameras is proposed to enhance the accuracy and adaptability of the robot scene recognition in different home environments,which provides a guarantee for the accurate family scene recognition required in the process of object search.Firstly,a family scene recognition model based on RGBD camera and fish-eye camera is constructed to enable the robot to obtain the required complete scene visual information by sufficient observation field.Secondly,a fish-eye scene image recognition model based on selective feature fusion is designed to improve the recognition efficiency and accuracy of fish-eye scene images.Then,by means of model calibration and correction,a reinforcement and growth method of robot family scene recognition ability is constructed,which not only enhances the ability of the robot scene recognition in the current home environment,but also equips the robot with the capacity to recognize new categories of scenes.The experimental results show that the proposed method outperforms the current mainstream methods in terms of the accuracy of family scene recognition and achieves accurate scene recognition of the robot in different home environments.(4)Aiming at the problem that the observation direction of the robot affects the accuracy of family scene recognition,an active family scene recognition method based on autonomous selection of observation direction is proposed to improve the family scene recognition performance of the robot during the object search.Firstly,an action decision model based on deep Q-learning network is designed to enable the robot to make reasonable motions to change the observation direction in order to obtain the family scene images from different observation directions(i.e.multi-view family scene images),which are beneficial for recognition as well.Secondly,a deep learning-based multi-view family scene recognition model is constructed to achieve accurate recognition of multi-view family scene images by building a scene scoring model and a scene score processing module.Then,in order to train the multi-view family scene recognition model,an autonomous generation method of the multi-view family images is designed.The experimental results indicate that,compared with the current mainstream scene recognition methods,the proposed method realizes that the robot can actively select the appropriate observation direction and accurately recognize the family scenes,so that the robot can obtain accurate scene recognition results in different locations of the complex family environment.(5)Aiming at the problems of low model training efficiency and poor performance of the existing active object detection methods based on reinforcement learning,an active object detection method based on behavior cloning and prediction revision is proposed in this dissertation,which achieves accurate and efficient active object detection by the robot in the object search process.Firstly,the idea of behavior cloning is used to transform the problem of active object detection into an action classification problem,and a multi-input action classification model for active object detection is designed.Secondly,an automatic expert data generation method is constructed to provide sufficient training data for the action classification model.Then,a model prediction revision method is designed to improve the success rate of the robot active object detection task by correcting the incorrect actions during the task execution.The experimental results show that the proposed method has advantages in terms of accuracy and efficiency compared to the current mainstream active object detection methods,and the effectiveness of the proposed method is verified in practical application scenarios.