Research On Cross Domain Sketch Retrieval Method Based On Deep Learning

With the increasing popularity of electronic touch screen devices such as mobile phones and tablets,the rendering and acquisition of sketches has become easier,and its related research has attracted more and more attention.Among them,sketch natural image oriented cross domain retrieval(abbreviated as sketch retrieval)technology has become one of the current research hotspots in the field of computer vision.Sketch retrieval technology plays an important role in the field of intelligent buildings and security,helping designers quickly retrieve building cases and compare feature information between security personnel,improving efficiency and accuracy.Although sketch retrieval has received a lot of research,it still faces some difficulties,mainly including: sketch features are more difficult to extract effectively than natural image features;The production of sketch data sets is not standardized,and the number of sketches in the data set is insufficient;The domain differences between sketches with the same semantics and natural images are too large.In response to the above problems,this thesis designed a feature extraction neural network for sketches,and then designed a triplet neural network that uses a triplet loss function to narrow the domain differences between sketches and natural images.The triplet neural network was embedded in a hardware platform based on the raspberry pie to achieve practical applications of sketch retrieval.The main research contents are as follows:(1)To solve the problem of insufficient images in the sketch data set and difficulty in extracting sketch features effectively,this thesis adds extended convolution and depth separable convolution to its network structure based on Sketch-a-Net to achieve effective extraction of sketch features.Then,based on this,a Sketch Fusion Network Model(SFN)was designed,and experiments were conducted on TU-Berlin,TU-Berlin Extended,and Sketchy datasets.The results show that SFN is effective in extracting effective features from sketches and alleviating the problem of insufficient collective sketching data.(2)To address the problem of excessive differences between sketch and natural image domains,this thesis constructs a triplet network based on SFN,Inception Res Net V1,and Mobile Net,respectively,and uses triplet loss to measure the consistency between sketch and natural image.Experiments on CASIA-Web Face,CUFS,and TU-Berlin Extended datasets have shown that this method effectively reduces the inter domain differences between sketches and natural images.(3)Based on the proposed algorithm,this thesis designs a sketch retrieval system using a SFN based triple neural network model.This system uses the Raspberry 4B as the core processor,and can achieve three functions: sketch data collection,sketch face retrieval,and sketch object retrieval.It has done preliminary work for the development and application of mobile sketch retrieval,and provided a feasible technical approach.Figure [43] table [13] reference [70]...