Environment Recognition And Semantic Understanding For Mobile Robots

Environment perception is a key component for mobile robot.As the demand of intelligent robot increases,research on environment perception has paid more attention on the semantic infor-mation,including object recognition,environment attributes analysis,end-to-end control learning,etc.Recently,the development of deep learning technique has significantly increased the perfor-mance of semantic perception,both in the accuracy and the methodology.However,most of cur-rent methods are built on closed experimental conditions.In regard to the openness and complexity of scenarios in the real world,there are still multiple challenges,such as the increase of percep-tion dimension,the large number of recognition classes,as well as the limited dataset.They have blocked a wider application of mobile robot.Under this background,this paper conducts percep-tion experiment in both environment recognition and semantic understanding tasks.Specifically,our contributions can be stated as follows(1)A one-shot 3D object detector with hierarchical multi-modal feature fusion is proposed in regard to the huge computation cost and the complexity of searching space.The method aims to increase detection efficiency as well as accuracy.The detector incorporates appearance feature in RGB image and geometric feature in depth image which are complementary in a hierarchical manner.It compensates the information lacking for single sensory unit.Then the prediction is performed in multiple feature layers to solve the scale problem caused by camera projection.By computation in 2D and matching in 3D,this method achieves real-time performance on multi-class multi-view 3D object detection.(2)An open-set semantic labeling method is proposed by utilizing conflict between differ-ent classifiers to recognize "unknown" class.It is in view of the limited known class number for complete environment labeling,and brings the semantic labeling task from closed-set to open-set to increase robustness.The method relies on a Conditional Random Field(CRF)to capture inher-ent spatial relationships between objects,and a Probability of Inclusion Support Vector Machine(P?SVM)to estimate the unknown possibilities for each known training class.The probabilistic out-puts from both CRF and P?SVM are then combined with the Dempster Shafer theory for conflict measurement and unknown rejection.The model has achieved better performance under various number of training classes.(3)In regard to the problem of lacking definition of road affordance as well as related dataset for scenes without traffic signs,a weakly supervised affordance inference framework is proposed based on robot trajectories.The method utilizes velocity to infer visual annotation,which consists of two steps.The first step adopts a Bayesian nonparametric model to segment vehicle trajectories into analytical vehicle actions,which can be projected on image to get partial affordance annotation.The second step implements a multi-task network of " TraceNet" to learn from the partial affordances and manages to predict complete road affordances.This method avoids manual annotation and can be generalized even in previously unseen scenarios.(4)In view of the high demand of localization precision caused by the separation of percep-tion and navigation,a visual navigation method is proposed based on GPS-level localization and publicly available route planner.The method fused two key traditional sub-tasks of perception and navigation by training a generative adversarial network to output an intermediate driving intention region.It follows both road structure in image and direction towards goal in local route,which can be fused with reliable obstacle perception to render a navigation score map for motion planning.The proposed method has avoided the need of traditional localization,and at the same time,has kept both the end-to-end learning performance and the modular flexibilityFor all the above methods,comprehensive experiments are conducted on various public datasets both quantitatively and qualitatively.The experiment results validated the effectiveness of pro-posed methods,which have brought performance enhancement in multiple perception tasks for mobile robots.