Image Smoke Semantic Segmentation Based On Deep Learning

Fire is one of the common and frequent natural disasters,and has caused very serious damage to society and nature.It is of great practical significance to study fast and effective fire detection methods.Since smoke is more observable,smoke-based fire detection methods are the key techniques in this field.Smoke object semantic segmentation has a wide application background in the field of computer vision and security monitoring.It can be directly embedded in a fire detection system to assist fire alarming tasks,which has a good warning effect on the follow-up personnel evacuation and property transfer.Therefore,it can be used as the underlying support technology of fire safety management and emergency management.Image smoke semantic segmentation based on deep learning can obtain exact locations of fire in a more intelligent way in realtime.Its mornitoring range is wider,the alarm speed is faster,and it can greatly reduce investment in human resources.However,smoke objects have the attributes of fuzzy,translucent and non-rigid.Therefore,how to obtain effectively key features from the smoke image and use these features to perform semantic segmentation on the smoke object are the main problems that needs to be solved urgently.This paper focuses on key problems of smoke semantic segmentation task,and combines the characteristics of the smoke to explore the feature extraction and semantic segmentation methods based on deep convolutional neural networks.The main work and contributions of this paper are as follows:(1)A method based on graphics is proposed to construct smoke image semantic segmentation datasets.Smoke has very fuzzy and erratic boundaries,it is very difficult and time-consuming to make full image annotations with manual methods.Therefore,aiming at the difficulty of obtaining smoke pixel-wise ground truth,this paper adopts the smoke simulation algorithm based on graphic tools to generate very realistic pure smoke renderings without background,and synthesizes them with a variety of complex backgrounds to generate training data.Threshold-based method is adopted to process the pure smoke renderings to obtain the ground truth corresponding to the training images.In addition,in order to increase the diversity and smoke patterns of the training data,and to some extent alleviate misclassification caused by intra-class dissimilarity,a data augmentation method is proposed,which mainly increases the diversity of the datasets by changing the color of the smoke.(2)This paper proposes a smoke semantic segmentation framework based on a dual-path asymmetric encoder-decoder structure.To overcome the dramatic changes in the texture,color,and shape of the smoke appearance,a dual-path frame containing coarse and fine paths is proposed.The rough path is an asymmetric encoder-decoder fully convolutional network structure with skipping layers,which is mainly used to extract the global context information of the smoke and obtain the coarse segmentation results of the smoke object.The fine path is an asymmetric encoder-decoder FCN structure that is slightly shallower than the coarse path to preserve the fine spatial details of the smoke.Finally,a fusion layer is used to supplement the spatial detail information to the coarse segmentation result to improve the accuracy of the algorithm.(3)A smoke semantic segmentation framework based on multi-scale and weighted multi-prediction loss is proposed.To solve the scale sensitive problem of the smoke,the algorithm proposes a multi-scale residual module,which expands the network in depth and width,and uses a bottom-to-up approach to fuse multi-scale information from different stages.In order to make the network converge more quickly and stably,the output of multiple intermediate layers of the network are also involved in supervised learning,and an adaptive weighted multi-objective joint training loss function is proposed.It is proved from the view of the back-propagation algorithm that the proposed loss function can work faster than those with single-objective loss function.(4)This paper proposes a smoke semantic segmentation framework based on multi-task gated recurrent networks.To improve the ability of network learning longrange context-dependent information of the smoke,the algorithm proposes an attention convolutional gated recurrent unit.For the inconspicuous smoke object in the image,a multi-scale context contrasted local module is proposed.By obtaining the context information of objects with different sizes and local information relative to context information,the ability of the network to recognize inconspicuous objects is significantly improved.To alleviate the problem of inter-class similarity,the algorithm proposes a dense pyramid pooling module,which can well aggregate the context information of different sub-regions.Through the introduction of the global context prior,the representation ability of the network to distinguishable features is improved.In order to further reduce the possible false segmentations caused by inter-class similarity,a classification branch is introduced to explore the internal relationship between semantic segmentation and recognition tasks.This branch can extract the highest-level semantic information of the image for assisting the semantic segmentation branch to significantly reduce the false segmentations caused by similar appearance.(5)A smoke semantic segmentation framework based on edge-reinforced and attention mechanism is proposed.In order to enhance the edge information of the segmentation results,the algorithm integrates semantic edge detection and semantic segmentation tasks into a framework,and uses the proposed attention mechanism to fully study the correlation between two subtasks.To obtain long-range contextdependent information in deep neural networks,the proposed self-attention mechanism is used for the high-level features of the object to improve the distinguishability of the obtained features and solve the problem caused by intra-class dissimilarity.Meanwhile,with the supervision of semantic edge,the inter-class similarity is alleviated.Finally,aiming at the severe imbalance between positive and negative samples in the sematic edge detection task,an edge loss function is proposed to put more attention on difficult samples.By continuously improving the network structure,loss function,and proposing the targeted functional modules,the four semantic segmentation frameworks proposed in this paper have overcome many difficulties in the task of smoke semantic segmentation and gradually improved the prediction accuracy of the algorithm.