Research On Image Stitching Based On Object Perception In Dynamic Scene

This thesis originates from the the crosswise tasks related to the deep sea video communication in the key special project of the national key research and development plan ”Deep-sea key technology and equipment”.One of the main task of this subject is to use panoramic image stitching technology to stitch images from different angles collected by multiple cameras with fixed position on the lander to obtain video images with panoramic angles in the seabed.Due to the limitations of the device,the focal points of different cameras cannot be fixed at the same position,so the constraints of the homography transformation model cannot be satisfied between the imaging planes of different cameras.During the research of the subject,after many experiments,it has been found that although the existing algorithms can obtain better stitching results in static scenes,serious misalignment will occur in dynamic scenes,at the same time,the movement of marine lift between the cameras will cause ghosting and other errors in the stitching results,it is difficult to get ideal results.In order to solve the problems in the research process,by analyzing the changing characteristics of the dynamic scene content,an image stitching algorithm based on object-perception is proposed in this thesis.The main work of this thesis:(1)A method of stationary background modeling using Elastic Local Alignment technology is proposed,to complete the alignment of images with large parallax.Frame-by-frame alignment of continuous frame images using stationary background modeling results can ensure the consistency of the stitching results between frames,improve the image stitching efficiency under dynamic scenes,and reduce the impact of scene content changes on the image alignment results.(2)A local misalignment metric function based on Sigmoid function is designed to calculate the misalignment between corresponding pixel bits.Using the Sigmoid metric function for local misalignment measurement can reduce the sensitivity of the seam finding algorithm to local misalignment and avoid the fierce jitter of the optimal seam finding result between continuous frame,it is better for the human perception.(3)Introduceing the instance segmentation algorithm into the preprocessing process of the image stitching algorithm,and proposed a method to adaptive detect the saliency of dynamic foreground object.By improving the saliency weight of the object,it avoid the incorrect seam cut of the foreground object by the seamline,and solve the ghost phenomenon in the stitching result.Through the experiments on the image stitching algorithm proposed in this thesis,the results show that the algorithm is more robust to the continuous change of the scene content and the movement of the foreground object,can better maintain the integrity of the object instance in the scene.Compared with the existing algorithms,the algorithm proposed in this thesis has the best performance.