Research On Active Scene Reconstruction And Semantic Analysis

Scene reconstruction and semantic analysis are the main difficulties in computer vision,computer graphics and robotics.As the demand for digital geometry continues to grow in the industrial and entertainment field,modeling and analyzing scenarios is more important than ever.The rapid development of depth scanning technology in recent years has made the scene modeling more and more convenient.Due to the innovation of this technology,scene reconstruction and semantic analysis techniques have evolved from simple situation to complex and from low dimension to high.On the other hand,after years of development,mobile robots have gradually become practical.In the traditional fields of manufacturing,logistics,and service industries,there have been many examples of using mobile robots to improve production efficiency or replace manual operations.In the field of scene reconstruction and semantic analysis,the acquisition of 3D data is almost always done by manual scanning.For large-scale complex scenes,manual scanning is undoubtedly a very heavy task,which consumes a lot of manpower and material resources.In addition,due to the lack of proactive analysis and guidance in the data collection process,the collected data is often incomplete,thus losing the value of modeling and analysis.To solve above problems,this paper studies the robot-based active scene reconstruction and semantic analysis technology in the unknown environment.In the aspect of scene scanning and reconstruction,this paper studies the method of autonomous reconstruction of unknown scene.In the aspect of scene semantic analysis and understanding,this paper studies the active scene semantic segmentation technology.In the aspect of object analysis,this paper studies the method of fine-grained object identification in complex environment.The main contributions of the paper include:1.Robot autonomous navigation guided by tensor field: Currently,field-based path planning algorithms mostly use vector fields.However,the vector field has many problems in practical situation such as local minimal trap,which greatly limits its application.This paper introduces the tensor field into the field of robot navigation for the first time,and proposes an autonomous navigation algorithm based on tensor field guidance.Compared with the traditional vector field(e.g.potential field),the tensor field has the advantages of less singular points and no local optimal traps.The generated local path can inherently avoid obstacles,and is smooth and stable.In addition,the topology of tensor field can help robot to do global planning,which greatly improves the efficiency of scene scanning.Based on the tensor field,we implement an autonomous robot scanning system and test it both in virtual and real scenes.The experimental results show that the performance of our method is significantly better than existing robot-based scanning systems.2.Camera’s 3D scanning trajectory optimization for high quality reconstruction:The scanning trajectory of camera has a great influence on the quality of scene reconstruction.High-quality reconstruction requires that the camera’s viewing angle should be as perpendicular as possible to the surface of area,and camera must keep stable during scanning process.In addition,since the camera is fixed on the robot head or arm,the trajectory of camera also needs to meet the constraints of robot motion model.Considering above requirements,we propose a new camera’s 3D scanning trajectory optimization algorithm,which formalizes all constraint items into an energy equation and then solves the trajectory by minimizing the energy equation.The difficulty of optimizing the equation is that it is highly non-convex,and is difficult to obtain an analytical solution.To solve this problem,we subtly transforms it into a linear programming problem,which can be tackle easily.The optimized trajectory not only ensures high-quality reconstruction,but also greatly improves scanning efficiency.The experimental results also verify the validity of our method.3.Robot active scene perception and semantic analysis: The majority of existing works about scene understanding pursues passive analysis,in which scene understanding is conducted over already acquired RGBD sequences.Unfortunately,this often greatly limits scene understanding performance.To solve this problem,this paper proposes a robot active scene perception and semantic analysis method.We first propose a novel incremental semantic segmentation network,which performs online semantic segmentation.With the help of this network,we introduce the concept of semantic segmentation entropy,which is combined with geometric reconstruction entropy to compute the next best view.In order to improve the scanning efficiency,we optimize the continuous scanning trajectory between two NBVs.The method is tested on a large virtual datasets.Experiments show that the proposed algorithm can get high-quality semantic segmentation results and significantly outperform several existing algorithms.4.Fine-grained identification of single object in complex environment: It is difficult to identify the fine-grained class of a single object in complex environment.Most of the existing algorithms only use a single view image to do the job and often get poor performance.In this paper,we propose a 3D attention model aiming at the fine-grained object identification.The model can guide robot to collect data from multiple views.Based on multi-view information,we perform identification for the object.In order to extract useful features for object identification,we use a recurrent neural network to fuse multi-view images.To verify the effectiveness of the proposed algorithm,we implement a robot-based active object identification system.The results show that our system can identify target object accurately.