Semantic Mapping For Domestic Service Robots

The construction of intelligent robotic agents that are able to percept and understand their surroundings autonomously has been a long-standing goal of engineers and scientists in the field of robots with artificial intelligence. In re-cent years, with the advances of research on domestic service robot and the is-sues of novel depth sensors, building3D maps of indoor environments using the RGB-D cameras attracted more and more attention and investigation. Particu-larly, achievements from many areas, such as robotics, computer vision, computer graphics and so on, are utilized to align and merge RGB-D image sequence and further obtain the global represent of3D scenes. There are many in-depth inves-tigations and valuable works in the aspects of precision, scale and speed of map building and map representation of real indoor environments.On the require of3D map models for indoor mobile robots, We systematically studied the map building approaches with RGB-D cameras in this thesis from the perspective of autonomy, and has constructed an automated information process-ing chain from sensor hardware to high-level semantic model. In this context, autonomy means that the3D maps of the indoor scene are built by robots auto-matically, and the maps require automatically analysis and understand to gener-ate semantic model. It is a low-cost environment modeling technique to obtain three-dimensional semantic model of indoor environment with RGB-D cameras. The study of this technology has extensive application value and important com-mercial value. For example, the pure RGB-D camera navigation derived from this technique is a hopeful alternative to the current mainstream but expensive laser based navigation.In this paper, the following three aspects are taken into consideration to achieve the autonomy. Firstly, the robot should be able to scan the scene and capture RGB-D image automatically. It is one of the issues addressed by this article that how to plan the scene scanning trajectory of the camera automatically on limited priori information about the scene, and generating a sequence of images which meets the mapping system requirements. Secondly, autonomy requires that the mapping system should be robust enough. It should be able to adapt to real scenes with various texture and structures and build a global map successfully, even without manual intervention. In addition, one of the motivations of this article is to make the robot build3D map of large-scale scene online and treat the robot as an autonomous agent to percept and understand its surroundings, so real-time performance and scalability is also the pursuit of this article. Thirdly, the3D maps of scenario need to be interpreted into semantic maps that the robot can understand.A new schema that robots build indoor semantic maps autonomously is pre-sented by investigating the above three sub-problems. Specifically, the main work of this paper is divided into the following three aspects:First, for the automatic image acquisition problem, we defines such two basic actions as the rotating scanning and mobile scanning, and obtain action sequences of scanning the scene on the existing2D grid map which is regarded as prior information of the environment. To gain the best scan plan, a gain function is also defined to evaluate the merits of scan plan and the optimal solution is achieved by random search techniques.Second, for the RGB-D mapping, to obtain globally consistent3D maps, we extract key frames from RGB-D image sequence, and spatial alignment, loop clo-sure detection and global optimization are taken on the key frame sequence. We have a deep investigation on the alignment and loop closure problem on RGB-D frames in order to achieve robust and rapid mapping performance. In addition, we adopt method presented in KinectFusion to achieve elaborate surface reconstruc-tion, and discuss the storage problem while extending the method to large-scale environment.Third, as for scene analysis, a fast plane detection algorithm is presented for the point cloud of the scene. Segmentation of the point cloud is done by extracting the planes in the scene, and then extracting features from separated parts of point cloud and recognizing with simple rules of the indoor environments. As a result, unsorted points set is converted in a3D topological map with semantic information.The contributions and innovations of this paper include the following three aspects:Firstly, a RGB-D camera scan-path planning method is proposed for3D mapping in this paper. The input images of current RGB-D mapping system are captured mainly by hand-held camera traversing throughout the environment. It suffers from low degree of automation, especially collecting images for large-scale scenes. And though this method, the camera scanning trajectory can be achieved in an automatic way. Experiments show that the automatically planned scan trajectory is very similar to the one designed by the expert, and thus prove the effectiveness of this method.Secondly, this paper presents an extremely robust and fast point and plane features based RGB-D image alignment algorithm. We perform frame-to-frame alignment experiment on adjacent frames in the key frame sequence (close to3,000frames), and the result of none error demonstrates the robustness of this algorithm. The algorithm avoids the time-consuming ICP-style alignment techniques, so it is of high efficiency and can achieve real-time performance on mainstream PC with-out GPU acceleration which in turn guarantees the robustness of the automatic mapping system. In addition, the combination of rapid feature matching based loop closure detection and global optimization techniques, and coupled with the automatic scan planning method described earlier, constitute a3D online mapping system running on the robot.Thirdly, a rapid plane extraction algorithm based on the projection transform is presented for the understanding of the scene through its point cloud model, it just cost a few seconds to detect and extract the planes from scene containing millions of3D points. The semantic map is obtained by subsequent scene seg-mentation and simply recognition of the scene.In this paper, we have designed a system which is able to automatically build3D semantic maps of indoor scene using cheap RGB-D cameras. It should be noted that the system still needs a2D grid map as the prior knowledge of the scene. Therefore, in practical applications, it still cannot be completely departing from the mainstream LRFs based perception and navigation. Getting rid of the prior knowledge of scene and fulfilling the task of exploration and semantic mapping of the unfamiliar environment just using an RGB-D camera, will be an important direction for future work. Though lots of works need to be done to realize the practical full autonomous and pure RGB-D camera navigation and perception, this work can still be regarded as an important step toward this promising goal.