Research On Regular Spherical Image Representation And Its Wavelet Transform

With the great-leap-forward development of virtual reality technology,automatic driving,unmanned air vehicle,robot,and other technologies,spherical image is emerging as a new media form.Spherical images can provide more information than planar images and bring viewers an immersive feeling through stronger reality sensation and interaction.At present,the research on methods and techniques of spherical image segmentation,coding,and recognition has become a hot research topic.All the spherical image studies face a core problem-the representation of spherical image data.Spherical image data through the projection often expressed as a plane image.However,a sphere is a two-dimensional manifold whose embryo is different from a plane.Such planarizations break the consistency and continuity of the spherical data.This inevitably leads to the following questions: 1)to any kind of plane-based representation,any continuous spherical image theoretically have an infinite number of representations;2)projection can introduce distortion and redundancy.These problems hinder the spherical image processing and analysis and further the development of spherical image processing technology.The successful application of traditional image processing technologies on large scale benefits from the regular representation of plane images and the efficient digital image processing technologies also depend on such a representation.However,for spherical images,such effective regular representations have not been well developed,making it challenging to implement basic operations such as filtering and up-sampling,and sub-sampling.It is impossible to construct complex processing tools such as spherical image transformation either.Therefore,this dissertation studies the regular representation of spherical images,corresponding indexing schemes,the generation of spherical images,and the spherical image analysis tools.In this dissertation,a regular representation of spherical images is proposed to maintain the spherical topology.Based on the new image representation and its regular structure,indexing systems with a linear structure for the representation are established to realize high effective retrieval of spherical images.A resampling algorithm based on the spherical measure is also proposed to generate high-quality regular spherical image representation.We also construct a spherical image wavelet transform based on the linear structure index system proposed in this dissertation using the lifting wavelet framework.Such a spherical wavelet provides a multi-scale analysis tool for spherical image processing.The work of this dissertation mainly focuses on the following aspects:First,a regular spherical image representation is proposed.The traditional representation methods such as ERP,Cubemap,Octahedron,and ACP are based on planar projection.However,the planarization method causes severe distortion and redundancy,which will destroy the consistency of spherical image sampling and the continuity of viewpoint.To eliminate the defects caused by the projection process,we can construct spherical representation models directly on the spherical surface with the spherical lattice’s aid.A spherical triangular grid can divide the sphere into uniformly distributed curved triangular regions with an approximately equal area.It is more fit for the characteristics of the sphere than other spherical grids,such as quadrilateral grids and hexagonal grids,and is more conducive to numerical calculation.Therefore,in this dissertation,the spherical image pixel is modeled using the spherical triangular grid directly defined in the spherical space,and its formal representation of the spherical triangular pixel is also given.The model preserves the spherical measure of the spherical image data and forms an approximate normalized representation based on the spherical finite discrete space.In order to further study the characteristics of spherical triangular pixels,this dissertation constructs a SLIC superpixel segmentation algorithm for spherical images based on this regular representation.The algorithm is applied directly to the simply reconstructed spherical triangle pixels and can generate spherical superpixel regions with reasonable shape and closed contour.Both the subjective and objective results of the superpixel region generated by the superpixel segmentation algorithm are superior to the traditional SLIC algorithm directly applied on the spherical image after projection.The experimental results show that the spherical triangle pixel model of the regular spherical image representation can directly reflect the spherical measure and effectively eliminate sampling redundancy and correlation destruction caused by the planar representation.Second,an efficient regular indexing scheme for spherical triangular pixels is proposed.Traditional addressing methods for the triangle mesh can be used to retrieve spherical triangle elements of regular spherical image representation.However,such addressing schemes can only build an address system with a nonlinear but not linear structured indexing system.As a result,the system can’t retrieval image elements randomly using a linear structure like plane image and cannot be directly used for filtering,up-and sub-sampling.It is not conducive to establish a spherical image transform complex spherical processing tools.Therefore,under the regular spherical image representation framework,this dissertation analyzes the relationship between spherical triangular pixels and establishes an indexing scheme based on barycentric coordinates and a binary indexing scheme.The barycentric coordinate based index is based on barycentric coordinates of the triangle.It uses two-dimensional linear integers to describe the topological relations of the vertices of the spherical triangle pixel in the spherical domain.However,this index system organizes pixels in an equilateral triangle manner,and its index space is filled with a large number of invalid regions,which need special treatment.In order to solve this problem,this dissertation further proposes a binary indexing scheme.The indexing scheme maps the spherical triangle pixels to a continuous right triangle in the plane space.It is similar to the plane image index and can maintain the original topological relationship of the spherical triangle pixels,which is convenient for convolution filtering,up-and sub-sampling,and also provides a foundation for the subsequent construction of the spherical wavelet transform.Third,a new method of regular spherical image generation based on resampling is proposed.Massive spherical images are given in ERP format,and other formats can be generated by converting from ERP format.How to transform spherical images in ERP format to regular spherical images is the crucial problem of regular spherical image processing and is also the basis of establishing subsequent spherical wavelet transform.For the regular spherical image representation,finding a high-precision resampling algorithm for triangular pixels is the key to generating a high-quality normalized spherical image.In this dissertation,based on the regular spherical image representation,two kinds of resampling methods for spherical images involving spherical radial basis based and spherical measure based resampling are proposed.The first kind of resampling methods includes a basic algorithm and a fast version.Under the current index system,the basic algorithm is to calculate the regular spherical image pixel value with a spherical radial basis function constructed on the spherical domain as the interpolation function.The resampling performance of this algorithm is better than that of traditional planar interpolation algorithms because the basis function’s weight can be accurately obtained from the geodesic distance.However,the local support region of the basis function is complex and the location of the reference point is variable.It increases the determination of local support constraints,leading to low efficiency.Therefore,a fast version of the radial basis function resampling method is proposed.By predicting the neighborhood relationship between the local support region and the spherical triangle grids using the triangle barycentric coordinate based indexing,the algorithm can significantly reduce the local support constraint determination operation and greatly improve the computational efficiency.However,the method based on spherical radial basis functions ignores the regular spherical structure and the triangle pixel shape characteristics.We then transform the spherical interpolation problem to a spherical integral problem and deduce another resampling method based on the spherical measure by establishing the mapping relationship between the spherical and the parameter domain through the spherical measure.This method has a closed-form and superior to the basic spherical radial basis function method in terms of time efficiency and reconstruction accuracy.We also verify the performance of two kinds of resampling methods to generate regular spherical image representation by experiments for continuous synthetic spherical images and natural spherical images.The experimental results show that the resampling method proposed in this dissertation can effectively reduce the conversion error.The generated spherical images are superior to the traditional resampling method in both subjective and objective quality.Fourth,the spherical wavelet transform based on the regular spherical image representation is proposed.It is well known that lifting scheme provides a flexible and effective way to construct wavelet transform.Since the complexity of spherical structure is much higher than that of planar structure,spherical image pixels cannot be divided into even and odd elements as simple as that of planar image.Fortunately,the regular spherical image has a natural multi-resolution property,which perfectly matches the wavelet dyadic scale structure.In this dissertation,the wavelet transform of the regular spherical image is constructed by using the lifting scheme,which provides a multi-scale analysis tool for the spherical image.By using the dyadic indexing,this transform establishes a spherical wavelet decomposition tree and implements the decomposition and reconstruction algorithm in the spherical image wavelet domain.Like the wavelet decomposition tree of the plane image,each layer of the spherical image wavelet decomposition produces one low frequency subband and three high frequency subbands.In order to obtain a more compact spherical wavelet transform,filter bank constraints such as normalization and energy conservation are introduced in this dissertation to construct high-and low-pass digital filter banks.In order to verify the effectiveness of the proposed wavelet transform,the energy concentration degree of the spherical wavelet is explored through experiments,and the compact characteristics of the wavelet are verified.Besides,edge detection experiments are carried out to further verify its ability to capture geometric features of spherical images.