| One of the main problems to be solved in deep space exploration,military reconnaissance,remote sensing aerial photography and other imaging applications is how to rapid convert the high-resolution images which taken by multiple sensors with some overlapping regions to a full scene image with wide field of view.Wide-field-of-view high resolution image mosaic technology is an interdisciplinary technique in many fields such as optics,computer vision,image processing,and parallel computing.It has important theoretical significance and practical application value,and it has also become an international research focus.Wide-field-of-view high resolution image mosaic technology involves multi-source image registration,image fusion and stitching,edge repair,rapid parallel processing hardware platform design and other technologies.Considering the practical application,there are uncertainties in the number,type and position of image sensors,and the throughput of the output image data is huge,moreover,the application environment of the system may be affected by the changes in light and image content,and the interference of parallax and moving object.All of them require higher requirements on the processing accuracy,robustness,universality of the algorithm and the computational efficiency of the hardware platform.This paper focuses on these key technologies of wide-field-of-view high resolution image mosaic technology,and the main research content and innovations are as follows:Firstly,a high-precision robust fast image registration method for complex situations is proposed.The method improves the ORB(Oriented FAST and Rotated Brief)algorithm for detecting feature points in the image to be registered.By increasing the preprocessing of the scale images,the quality of the extracted feature points is improved and the algorithm is scale-invariant.Based on the distribution of feature points,the constraint conditions of feature point distance are given for removing a large number of unreliable matching point pairs.In addition,a feature point matching algorithm based on geometric constraints is proposed to deal with the situation of image with sparse feature points and textures.The feature descriptors based on geometric invariant are constructed for the feature points,and the matching cost function is updated to evaluate the similarity between feature points.Finally,the improved sample consistency algorithm is employed to obtain an accurate geometric transformation between the images.The results of image registration for various complex scenes show that the proposed algorithm can significantly reduce the false matching rate while retaining more high-quality matching point pairs,which has strong robustness and convenience for fast processing.After that,a low distortion multi-view image mosaic method with content protection is proposed.The method automatically sorts the sequence images,and the stitching method based on optimal suture line search is utilized to complete the stitching without the parallax interference.Then,the method of drift correction is applied to the stitched image.For the image stitching with parallax,the feature point screening mechanism,the bundle adjustment method and the optimized local transformation model are used to eliminate the parallax effect,thus overcome the disadvantages of the traditional image stitching methods based on global homography which can only handle the images that approximate 2D planes.In the case of large distortion in the non-overlapping region of the image,a stitching method based on multi-constraint local hybrid transformation is proposed to maintain the alignment precision of the overlapping region while relieving the shape distortion of non-overlapping region.The experimental results show that the proposed method can effectively deal with the image parallax problem,and the stitching misplacement and the ghost problems are eliminated.In addition,the shape distortion is significantly reduced,and the main content and overall structure of the image are protected.To solve the problem that most of the traditional image restoration methods can only repair the voids in the middle area of the image,a method for repairing the edge damaged image with wide field of view is proposed.This method adopts an algorithm based on the optimal seam search to fill in the image content on the lowest energy of the image.It avoids the regions with rich details,and it fills single-pixel-width slits by copying and interpolation to stretch the image to the regular boundary.Finally,the energy function constraints and mesh optimization method are utilized to adjust the local grid image.The experimental results show that the proposed method can restore the damaged edges while protecting the original image content and the overall structure,thus the wide-field stitching image with low-distortion and regular shape can be obtained.A new fast image processing hardware platform based on Tilera many-core architecture is designed for stitching the high-resolution images with wide field of view.The limitations of traditional multi-core hardware processing platforms based on CPU/GPU and the embedded hardware platforms such as DSP/FPGA are analyzed.According to the application requirements of fast stitching images with high-resolution and wide-field-of-view,a CPU core board based on the Tile Gx64 many-core processor and an extended backplane of functional interface are designed.The method of this paper is designed in parallel by data division and task scheduling,and the overall performance and operating efficiency of the system are optimized and improved.The experimental results show that the designed hardware system can able to stitch the images quickly and efficiently,and the performance of processing efficiency and power consumption are significantly better than the traditional CPU.Finally,the paper summarizes the completed work and innovations,and it prospects the future development and research direction. |
PDF Full Text Request