Pattern Semantic Information Guided Shoeprint Image Retrieval

Shoeprint is one of the valuable clues in the crime scene.Retrieving similar shoeprints in previous cases can narrow the scope of the case and even target the criminal suspect.Shoeprint is of great significance for the investigation of the case.Because the crime scene is changeable and the suspect’s shoeprints are mostly left behind in a very tense and hurried situation,the shoeprint image collected on the scene often have defects such as large-area incompleteness,complex background,excessive noise and interference.Among them,large-area incompleteness shoeprint image carries less semantic information,which increases the difficulty of shoeprint retrieval.What’s more,it is difficult to extract shoeprint image from complex backgrounds,which causes valuable clues not to be used to solve the case.Aiming at the above two situations,this thesis proposes pattern semantic information guided shoeprint image retrieval algorithm.Hand drawn supplementary shoeprint patterns are used to enrich semantic information and guide the generation of complete and clear virtual shoeprint images,so as to improve the retrieval accuracy.The main work is as follows:(1)A shoeprint image retrieval algorithm based on sketch guidance is proposed for largearea incompleteness shoeprint image.The shoeprint content model is constructed based on the Generative Adversarial Network.The dilated span convolution module is used to combine the dilated convolution with the span convolution,and the receptive field is expanded to extract more abundant features while down-sampling.The deconvolution interpolation fusion module is used to combine deconvolution with bilinear interpolation.The feature maps of different depths and scales are fused to enhance the generation ability and fitting ability of the model.The incomplete shoeprint image is supplemented by sketch to form a mixed image of scene pattern and sketch pattern.The image is input into the shoeprint pattern content construction model to generate the virtual shoeprint image with complete content and clear structure.Then,the virtual shoeprint image is retrieved by the Fourier Mellin transform based shoeprint image retrieval algorithm.Experimental results on two public available datasets show the proposed algorithm is very competitive on retrieving large-area incompleteness shoeprint image.(2)A sketch-based shoeprint image retrieval algorithm is proposed for shoeprint image which is difficult to extracted due to complex background.The shoeprint semantic information model is constructed based on the Generative Adversarial Network.The multiscale dilated convolution fusion module is used to combine the feature maps of different receptive fields to enhance the feature representative ability of the model.The deep and shallow deconvolution residual fusion module is used to combine deconvolution with residual operation to prevent gradient disappearance and gradient explosion on the basis of enhancing the network fitting ability.The shoeprint sketch drawn manually is input into the shoeprint semantic information construction model to generate the virtual shoeprint image with complete semantic information.Then,the virtual shoeprint image is retrieved by the Fourier Mellin transform based shoeprint image retrieval algorithm.Experimental results on two public available datasets show the proposed algorithm is very competitive on retrieving shoeprint sketches.The proposed algorithm has achieved good retrieving results compared with state of the art algorithms on FID-300 dataset and CS dataset.The cumulative matching score of the sketch guidance based retrieval algorithm at top 20% on FID-300 dataset reached 94.5%.The cumulative matching score of the sketch based retrieval algorithm at top 20% on CS dataset reached 87.5%.