Research On Image Semantic Understanding Based On Deep Learning

With the sharply increment of images on Internet,image retrieval has become the focus of research.It is prerequisite correctly understanding images for fast image retrievaling.After pulling an image with a person in,the image semantic understanding system we proposed descripting the people and scene inside the image by using a sentence.Traditional image semantic understanding method cannot combine image understanding with natural language processing in a properly way,which causes low efficiency and high time consuming.Deep Learning based on unsurpervised training and surpervised fine-tuning on millions of images is replacing the traditional method in the field of image processing for its perfect performence in image classification and recognition challenge,and recurrent neural network is becoming the main method in Natural Language Processing.Because of image sematic understanding at the begining study based on deep learning,there is few related work currently.By extending LSTM layer to the Neural Image Caption Generator which based on Deep Learning,we get an end-to-end image semantic understanding system DLNN,after pre-training and surpervised fine-tuning.Experiments results of pulling image with people into DLNN suggests that DLNN can understand people and scene efficently in either selfies,busts and images with whole people inside the image.BleU scores on public dataset has also proved that DLNN could be efficient in semantic understanding images.Further more,We reference the improvement of MLBL-F by improving DLNN to D2 LNN,which performes more better in description details of the image.By testing D2 LNN on the images which has tested DLNN,we find that D2 LNN and DLNN are both efficient in understanding people inside images,but the former one does better in understanding details of the images by showing it in a sentence.