| With the improvement of living standards,people's demand for more intelligent service robots is also increasing,but now the intelligent service robots are still far from meeting people's expectations.Simultaneous Localization and Mapping(SLAM)technology is the basis for the service robot to operate in the indoor environment.At present,great progress has been made,but the ability of the service robot to perceive and understand the environment is far from satisfying its long-term stable operation and the need for monitoring of environment,such as the perception of temperature in the environment,the understanding of objects,and the construction of maps in a dynamic environment.Therefore,this paper combines the RGB-D camera and the thermal infrared camera for the research on the construction of the thermal map and target detection technology of the intelligent service robot in a dynamic environment to improve the robot's ability to understand the environment.The specific contents are as follows:Pedestrians in dynamic scenes are detected and segmented to remove dynamic feature points and reduce their impact on dynamic scene mapping.Based on thermal infrared camera and Kinect sensor,this paper compares the end-to-end pedestrian segmentation algorithm based on convolutional neural network,and proposes a pedestrian segmentation algorithm based on the combination of convolutional neural network and temperature and three-dimensional information.The images obtained by the infrared camera and Kinect camera are used to detect the 2D bounding box of pedestrians,and then the pedestrians are initially segmented based on the temperature information,and then the preliminary segmentation results are mapped into three-dimensional space,using point cloud filtering and other algorithms to remove outliers,and finally achieve the pedestrian's precise segmentation.Based on the ORB-SLAM2 algorithm,combined with the proposed pedestrian segmentation algorithm,a 3D environment thermal map is constructed for the indoor dynamic environment.First,the RGB-D camera and the thermal infrared sensor are registered,and then based on the HSV color space,the color image and the thermal infrared image are fused to realize the representation of the texture and temperature information of the environment.Then combined with the pedestrian segmentation algorithm,ORB features of the dynamic pedestrians detected in the environment are removed to reduce the impact of the dynamic pedestrians on the pose estimation,and finally the thermal map under the dynamic environment is established.In order to improve the service robot's understanding of the environment,this paper constructs an object-level 3D semantic thermal map based on a 3D object recognition algorithm.First,YOLOv3 is used to detect objects in the environment,and then the obtained 2D bounding boxes are projected 3D space to get the points cloud.Then the obtained 3D points cloud is clustered and segmented to obtain the point clouds of the objects.Then,an algorithm based on the convex polygon is used to get the 3D size and direction of the object.Finally,combined with the thermal map construction framework,an object-level semantic thermal map is constructed,which contains the object's category,size and temperature information to improve the robot's ability to perceive the environment.Finally,this paper uses a mobile robot as an experimental platform.In each chapter,experiments are given to verify the effectiveness of the multi-modal pedestrian segmentation algorithm and the accuracy of the 3D environmental thermal mapping under dynamic scenes.It also verifies the feasibility of constructing a three-dimensional semantic thermal map based on 3D object detection. |
PDF Full Text Request