Research On Key Technologies Of Vehicle Re-Identification Based On Urban Video Surveillance Images

Vehicle re-identification is a technology that uses computer vision technology to determine whether a particular vehicle exists in an image or video sequence.It mainly solves the problem of identifying and retrieving vehicles across cameras across scenes,which is a sub-problem of image retrieval.In recent years,the vehicle re-identification method based on deep learning has achieved fruitful research results,but there are still some difficult problems affecting the effect of vehicle re-identification.For example,during vehicle detection,there are often problems such as false detection,missed detection,and repeated detection.When the monitoring scene changes,the performance of vehicle re-identification will greatly decrease.When the perspective of the vehicle used for matching is different,the visual appearance features of the vehicle will be very different.The vehicle license plate image is usually blurred and cannot be directly used for vehicle re-identification.When the acquired vehicle attributes are not rich,it will affect the result of vehicle re-identification,etc.Aiming at the above problems,this paper proposes a multi-model fusion for progressive vehicle re-identification framework based on urban video surveillance images,and proposes a series of methods and models from five aspects:vehicle image detection,cross-domain image style transformation,vehicle hidden perspective image generation,super-resolution reconstruction of license plate images,and vehicle appearance attribute learning.The effectiveness and robustness of the proposed method and the overall framework are verified by a large number of experiments on the urban surveillance datasets.The main work and contributions of this paper are as follows:(1)For the problems with existing vehicle detection methods,such as false detection,missed detection,and repeated detection,a fast vehicle detection method based on connect-and-merge convolutional neural network(CMNet)is proposed.First,a connect-and-merge residual block is designed,which is formed by assembling two residual branches in parallel with a connect-and-merge mapping:connect the input to the outputs of two residual branches separately(Connect),and merge the outputs of the connection as the input of the subsequent residual block(Merge),respectively.Then,a full convolution network with 4 branches is connected,and the 4 branches are respectively merged with the feature maps of the same scale in the connect-and-merge residual blocks to predict vehicle information,thereby achieving fast and accurate vehicle detection.(2)In the cross-domain scenario,the style of the vehicle image has a large change,and the performance of the model trained on the source domain images will be drastically reduced on the target domain.Therefore,a vehicle re-identification method based on transfer learning scenario adaptation(TLSA)is proposed.First,the vehicle images on the source domain are transferred to the target domain by using a transfer learning model based on generative adversarial network,so that the transferred vehicle images have the same style with the target domain images.Then,the deep features of the transferred vehicle images are extracted.Finally,the similarity between the two images of the vehicle image and the vehicle image database is calculated by the Euclidean distance to achieve vehicle re-identification under cross-domain conditions.(3)When the vehicle images of two different views(such as front and side)are matched,even if the same vehicle,the features of its visual appearance are quite different,so it is difficult to accurately distinguish whether it is the same vehicle.Therefore,a vehicle re-identification method based on multi-view image generation(MVIG)is proposed.First,eight different views of the vehicle image are generated for each single-view input image.Then,we extract the deep features of one original image and eight generated images respectively,and fuse the nine features to form enhanced features.Finally,by comparing the perspective-normalized vehicle features,effective vehicle re-identification is achieved.(4)Since the appearance of the vehicles has great similarities,such as the same brand,model and color,it is difficult to distinguish them only by the appearance properties of the vehicle.Therefore,a vehicle re-identification method based on license plate image super-resolution(LPSR)is proposed.First,the connect-and-merge residual network is used to detect and correct license plate images.Then,the low-resolution license plate image is converted into high-resolution license plate image using a super-resolution reconstruction technique based on generative adversarial network.Finally,a method based on license plate verification is used to match the vehicle image,and the similarity of the license plate image is compared to achieve accurate vehicle re-identification.(5)To quickly match moving vehicles,a vehicle re-identification method based on attribute and identity learning(AIL)is proposed,which acheves fast and rough vehicle re-identification by recognizing multiple attributes such as vehicle type,color,and more.Then on the base of this,a progressive vehicle re-identification framework with multi-model fusion(MMFP)is designed,which unifies CMNet,TLSA,AIL,MVIG and LPSR into one deep framework and uses a coarse-to-fine idea to achieve efficient and accurate vehicle re-identification.