Design And Application Of Image Cognition Platform For Mobile Inspection

Inspection means that the company assigns relevant personnel to evaluate the quality of the services provided by its dealers every quarter,and the dealers will then carry out rectification according to the evaluating results In recent years,mobile inspection has become a new trend in the industry Relevant personnel often upload images and texts through APPs to inspect their dealers.Taking the vehicle inspection system as an example,mobile inspection images are often uploaded only by professionals,so the size of image data in mobile inspection is relatively small.Although each image belongs to only one classification,the images of different classifications sometimes correspond to similar scenes,while the image descriptions are often not accurate enough,so there are great difficulties in image cognition The traditional inspection process mainly includes the following two problems:First,the automation degree of the inspection process is relatively low.Due to the large number of inspection items,the district managers often need to upload the inspection images and descriptions one by one during the inspection process,which will cost a lot of time.Second,the current platform does not comprehensively analyze the relation between images and text.The image description in the mobile inspection field often cannot accurately depict the information of the image.Dealers often refer to other stores' related services during the rectifying.The existing platform only supports querying according to the inspection category.It does not use the information of the pictures effectively,which is hard to meet the needs of users.At the same time,the semantic information of the text is not well considered during the natural language query.Therefore,the accuracy of natural language search is relatively low,and it is also hard for the final image archiving.Aiming at the above problems,this paper designs and implements an image cognition platform for mobile inspection.Based on the convolutional neural network and hash technology,we realize the service of image retrieval and image automatic matching.The long-short-term memory network is used to convert the image feature vector extracted by the convolutional neural network into the descriptions of the image,and then the image caption task is completed.The word vector technology is utilized to convert the text into vectors,and then we can provide the semantic search service by comparing the cosine similarity between the vectors.The main contributions of this paper includes:Firstly,an image cognition platform framework based on deep learning is proposed.The platform is divided into request handling module,text processing and image retrieval module,model training module and data persistency module,providing image retrieval and automatic matching,image caption and semantic search service.Secondly,the model of content based image retrieval and classification is designed.Aiming at the problem of low automation in the inspection process,we first use the image enhancement method to increase the image data.Afterwards,by modifying the VGGNet network structure,the hash technique is applied to the neural network.The hash value of the image is generated,and then the image retrieval service is provided by comparing the Hamming distance and the Euclidean distance between the images.We also utilize the output of the final classification layer of the VGGNet to match the image to the associated inspection itemThirdly,we design the method of automatically associating the image with text.As the lack of analysis between picture and semantics,we first preprocess the text data,and then the Encoder-Decoder structure is used to extract the feature vector of the image through the VGGNet as the input of the LSTM.Afterwards,we apply the beam search method to automatically generate text descriptions for the image,solving the problem of the low accuracy of image description.Finally,Doc2Vec model is used to convert the natural language into word vectors.By comparing the cosine similarity between vectors,we can provide semantic search services as well as the matching verification service between the text and inspection item,improving the automation of the entire processFourthly,we design and implement the image cognition system based on the above method.The Caffe and Tensorflow models were loaded through the open source API,and the images and model files were stored by OSS and HDFS,and then business data was stored using MySQL and Redis.