Research On Image Captioning And Application Of Coal Mine Scene

Image captioning is a cross-cutting task in the field of computer vision and natural language processing.Its goal is to give semantic interpretation of visual data and realize mapping from visual space to semantic space.The research of visual captioning technology has not only become a hot spot in academia,but also has received more and more attention in the industry,and has broad application prospects,such as the intelligent advancement of industrial video surveillance systems.At present,the coal mine video monitor system has been widely used in underground,but the abnormal scene is only identified by the monitoring personnel to observe the video,and then the countermeasures are taken to solve the problem,and the efficiency is low;the application scale is far beyond the scope of human observation,and the surveillance video is usually used for evidence collection after the event,so more hidden problems cannot be identified and dealt with at an early stage.It is of great significance to realize online intelligent monitoring of coal mine video monitor system through image captioning technology,which can greatly improve the safety management level and efficiency of event processing in coal mines.Because the visual features at the bottom of the image are very different from the semantic concepts at the top level,there are still many problems in the current image scene semantic description algorithm,such as determining the focus of the image,mining higher-level semantic information,and improving the detailed information describing the sentence.Therefore,this paper improves the above problems in image captioning technology,and proposes a image captioning algorithm based on Globallocal Feature and Adaptive-attention,extracting Global-local Features and introduce adaptive attention.The adaptive attention mechanism allows for more detailed information on the image and a more comprehensive and detailed semantic description of the input image.The research content of this paper is divided into the following parts:(1)Proposed a image captioning algorithm based on Global-local Feature and Adaptive-attention.In order to obtain more detailed information of the image,the local features and global features of the image are extracted in the coding part,and the local features of the input image are combined with the global features;the decoding part introduced an adaptive attention mechanism,adaptive allocation generates visual features and generated semantic word feature weights in the process of captioning,and outputs a more comprehensive and detailed semantic description text.(2)Introduced the experiment based on Global-local Feature and Adaptiveattention image captioning algorithm in public dataset.The experimental results of this paper are compared with the current mainstream image captioning algorithms.It can be seen from the results that our image captioning method describes the image content more comprehensively,accurately and in more detail.The description effect is better than other image captioning models and semantic descriptions are better.The image scene description system based on the Web front-end display is designed,so that the user can call the trained model to realize the semantic description of the online image scene through the browser client.(3)Introduced the application of the Global-local Feature and Adaptive-attention image captioning algorithm in coal mine scenarios.Obtain the video frame image of coal mine monitoring,and take pre-processing of key area image and denoising to make the captioning data set of coal mine scene;use the coal mine scene data set to train the image captioning model to realize the captioning output of the coal mine scene image;The semantic description system of the coal mine scene realizes the real-time semantic description output of the monitoring video on the underground coal mine,and makes an abnormal reminder for the abnormality.