Research On Image Matching And Super-resolution Reconstruction Of RGBD Images In Indoor Robots

The indoor service robot has been used extensively in many domains,such as education,entertainment,security,disability assistance and etc.Robot perception is the basis of indoor service robots and many researchers have paid their efforts in this field.One of the typical equipment of indoor service robot environment perception is laser devices.However,laser devices are very expensive which restricts the popularization of indoor service robots.Instead,RGBD Sensor?Such as Kinect sensor?are widely used for their cheapness.However,RGBD Sensor has a serious defect of low-quality images.In order to mitigate the problem,we purpose an indoor robot environment perception framework based on RGBD image.Our work mainly focuses on the misalignment between color and depth images,low-quality images,and low accuracy of geometric estimation.We take indoor scene as a case study and establish the indoor robot environment perception system based on the X2BOT wheeled robot platform.The experimental results show that our method performs better than existing approaches.The main content of the paper is as follows.First of all,two methods for the matching problem between color and depth images are proposed.First,we propose a Non-Parametric Depth Modification algorithm,which is able to obtain accurate depth and color images matching.We utilize the non-parametric Gaussian multiple model based on₂and built the regularization term to constrain the correlations between functions.Second,we propose an image matching algorithm based on multi-vector field constraint.A hidden variable is introduced to distinguish the inside point and outside point.Additionally,we build a vector field constrain,and calculate the probability value of all matches according to the Bayesian theory.Based on the probability value,we can obtain an accurate registration.Extensive experiments have been performed to verify the effectiveness of the proposed approaches based on self-built datasets and public datasets.Secondly,we propose a new image super-resolution method to improve the quality of images.Low-cost sensors sacrifice the image quality of color depth images,which results in inadequate precise feature points.The complex reconstruction algorithm increases the consumption of system power and reduces the robot's continuous operation time.However,low-cost,high-quality,low-power,and long-period are crucial for the survivability of robots.In this paper,we propose a super-resolution reconstruction algorithm to resolve the above problems.Our method fuses depth images and color images in image domain,and uses the color image to guide the super point reconstruction of depth image.Additionally,we use the residual fusion to build multi-source neural network learning algorithm.Our method can effectively mitigate the problems of the detail-loss and additional details in the process of reconstruction image.Extensive experiments based on public datasets showed that our method has high quality and high efficiency.Thirdly,we propose a new robust estimation method based on local constraint.We firstly establish initial correspondences by feature description matches.Then,we estimate the fundamental matrix and homography matrix using₂-LSC?short for₂with local structure constraint?and achieve refined correspondences.The₂-LSC is able to robustly deal with noise and outliers contained in point correspondences.Extensive experiments conducted on real images from public available datasets have demonstrated that it can achieve good estimation accuracy and superior performance over previous approaches,particularly when there are severe outliers.Finally,we establish the indoor robot environment perception system based on RGBD image.We relied on the X2BOT wheeled robot platform and established the indoor robot environment perception system.The performance of the system is analyzed from three aspects:?1?the accuracy of image registration;?2?the quality of images super resolution;and?3?the accuracy of geometric estimation.Experimental results showed the feasibility and superiority of the proposed environment perception system.