Semantic Analysis Of River Scene Based On Deep Learning

River aerial images contain information about water environment,coastal land environment,etc.At present,the aerial detection methods can not directly and accurately reflect the scene information.In this paper,the deep convolution network was adopted to automatically extract the features from river aerial images.And the image features were transformed into text sentences by the long-short time memory network,making the results more accurate and fast.And it provides the public with the intuitive river information.The research contents are following:(1)River scene video about 100 hours(4T)was collected by UAV,the ten targets and the corresponding evaluation standards for river scene detection were proposed,and river image datasets and caption datasets were labeled manually for subsequent experiments.(2)The frame selection target detection network was designed to locate and detect the targets in river scenes.And the network was optimized according to the object characteristics: The upper sampling layer was added to improve the location accuracy;some of the convolution layers were abandoned,and some normalization layers was added,which increase the detection speed.The adjacent position penalty item was added,which limit the unreasonable location results and improve the classification accuracy.(3)The language network based on LSTM unit was proposed,which can automatically learn the corpus dataset.By adding generalization layer and ReLu activation function,the language model generated more fluent and concise sentences according to English grammar.(4)The image networks and the language networks were composed to end to end semantic model according to the Encoder-Decoder form.To add the penalty item,the generated sentences can more accurately reflect the image content and complete the image caption task.The experimental results show that the proposed model can classify and locate 10 targets in the river scene.The average accuracy and IOU are 83% and 62% respectively,which are superior than tradition methods.The sentences from the end to end semantic network are fluent and brief,which can describe the river scene actually and simply.