Design And Implementation Of The Panoramic Camera System Basing On FPGA

The panoramic camera have a 360° angle of view horizontally and an almost 360° vertically,and it should have been widely used in the field of robotics,video conference,surveillance and virtual reality.With the panoramic camera,the robot can make a better decision in target recognition,path planning and obstacle avoidance.Because of processing mass data and the the complex image processing algorithm,it is necessary to meet the portability requirements.Almost all panoramas can not obtain panoramic images without distortion in real time,which l imits their application.Therefore,solving the real-time problem has become the focus of universities and corporations.Compared with other image processors,FPGA has the advantages of a higher speed,parallelism,lower power consumption and smaller size.Choosing the FPGA as the core component in panoramic camera can reduce the power consumption and size of the device.In this thesis,a multi-camera panoramic camera platform has been developed and the transformation matrix between the five images in the platform could be calculated in real time.The resolution of each image is 640×480 and the frame rate is 30 frames per second.There are several main parts in this study as following.Used by five wide-angle manual focusing CMOS camera module,the platform realizes the function of panorama camera.We choose FPGA chip to processing image in real-time.The image of the wide-angle camera is distorted.We get the internal parameters of each camera with Matlab software.Using the bilinear interpolation algorithm,we correct the distorted image in real-time by the look-up table.We registrate image by image featres.We extract the image featres u sing the SIFT algorithm,generate feature point descriptors by BRIEF algorithm and determine the matching points by minimum Hamming distance.To extract the image features and match these points in real,we optimize the image processing algorithms using data parallel,task parallel method,basing on FPGA We remove incorrect matches by RANSAC algorithm in parallel processing and obtain the transformation matrix.Finally,the transformation matrix parameters and the image data after correcting are transferred to PC to realize the image registration and image fusion.