Research On Point Cloud And Panoramic Image Fusion Algorithm Based On Contour

In recent years,virtual roaming has been favored by developers from all walks of life because of its strong sense of immersion,good interactivity,having the capacity to transcend space and time,and it is widely employed in the domains of education,healthcare,tourism,and other.Virtual tours are currently mostly implemented using panoramic photos and 3D modeling.Yet,based on using panoramic photos,the lack of three-dimensional information leads to weak immersion.The implementation method based on 3D modeling is time-consuming and laborious,and the laser scanning point cloud lacks rich texture information and has poor realism.Therefore,how to enhance the realism and immersion of virtual roaming is a hot issue in current research.Panoramic images contain rich texture information and point cloud data contain depth information,which have high complementarity,and the integration of point cloud and panoramic images is of great significance to improve the virtual tour experience.This paper aims at the fusion of 3D point cloud and image data of natural landscape,and conducts in-depth research on the problem of point cloud and image feature extraction and registration.It addresses the issue of difficulty in registration and fusion caused by the different expression methods of point cloud and image.The following is what this paper contributes and innovates:(1)The corrected panoramic image and 3D point cloud are uniformly represented in the spherical coordinate system,and the two are represented in the 2D Cartesian coordinate system coordinate system using the expanded grid of theodolite stations.The spherical model is used for projection transformation,and the correction of fish eye panoramic image and the conversion of 3D data to 2D image are realized;The spherical panoramic image and spherical point cloud image are expanded by using the longitude and latitude expansion network,so that the fish eye panoramic image and point cloud data have a unified projection coordinate system and a two-dimensional expansion method.Finally,the effectiveness of this method is verified through experiments.(2)This paper uses contour features to register point clouds and images,and proposes an improved point cloud and panoramic image contour extraction algorithm,which completes the contour extraction of panoramic images and point cloud projection images.Firstly,the color extraction method of HSV space is used to effectively remove the background interference of the panoramic image,and the outline discontinuity portion of the point cloud projection picture is filled in using the image morphological closure operation approach.Second,the point cloud projection picture and the panoramic image’s edges are extracted using the Canny edge detection technique,and then the contour extraction and contour tracking method is applied to track the contour line of the panoramic image and the point cloud projection image,and finally the contour noise information is removed by limiting the area of the minimum circumscribed rectangle of the contour,and the Gaussian filter is used to smooth the extracted contour information.The algorithm effectively extracts the contour information of panoramic images and point cloud projection images,and lastly uses studies to demonstrate the algorithm’s efficacy.(3)An improved mutual information contour registration algorithm is proposed.Firstly,the Hu invariant moment is introduced as the similarity measure between contours,and the contour similarity of the panoramic image and the point cloud projection image is compared by the sliding window,and the contour area with the greatest similarity is selected as the input of the registration algorithm.Secondly,the mutual information registration algorithm is adopted,and in order to get the transformation parameters for contour registration,the affine transformation is employed as the registration transformation model.Finally,the transformation parameters of contour registration are used to transform the panoramic image,which realizes the registration alignment of the panoramic image and the point cloud projection image.The algorithm effectively realizes the registration of panoramic image and point cloud projection image,and lastly uses a point cloud and panoramic picture fusion system to demonstrate the algorithm’s usefulness.