Research On Semantic Image Segmentation Based On Improved Svam

Automatic semantic image segmentation is a challenge task in the computer vision field,which is the key step to image understanding. However, users cannot understand or retrievean image by using raw image features extracted by image processing algorithms. Therefore,one of serious problems confronted by image segmentation and retrieval is the huge semanticgap between raw image features and image semantic acquisition.Selective visual attention model (SVAM) is the computational model provided formimicking the attention mechanism of human visual system, which can obtain the mostsalient region that attracts people greatly in an image. So, SVAM is able to implementsemantic image segmentation efficiently. On the other hand, as the main representative in thethird generation of artificial neural network, pulse-coupled neural network (PCNN) has thegood performance in image segmentation. In order to further improve the accuracy ofsemantic image segmentation, this dissertation devotes itself to automatic semantic imagesegmentation models or methods based on integration models of SVAM+PCNN. The majorresearch contents and conclusions of this dissertation are summarized as follows.(1) The output of the PCNN is a binary image, while the salient region detection result ofthe SVAM is a gray-scale image. Then, it is difficult to compare their performance of imagesegmentation directly and fairly. To solve this problem, this paper improves the receiveroperating characteristic (ROC) curve analysis method based on the same platform of theirgray scale image results transformed from their color image segmentation results.Experimental results show that the improved ROC analysis could evaluate different kinds ofimage segmentation models or methods efficiently. Another problem is how to prove that anintegration model is significantly different from its component models. To solve this problem,this paper introduces the mean square deviation in statistics and the statistic method ofStudent's t-test to the evaluation of different models. Experimental results indicate that thisindex and method can efficiently evaluate image segmentation models or methods.(2) Considering that the region of interest (ROI) extracted by saliency toolbox (STB)/Ittimodel is not large enough for semantic image segmentation, the integration model ofSTB/Itti+PCNN is proposed. This integration model took fusion image of color andorientation maps extracted by the STB/Itti model as the input image of PCNN, so that the strong image segmentation capability and the property of anti-noise of PCNN could beenhanced; Saliency map generated by STB/Itti was employed to identify the optimal iterationnumber of PCNN at once, and the PCNN did not require many iterations; the PCNN thendisplaced the WTA of STB/Itti to output the semantic image segmentation results.Furhermore, the image segmentation capability of the PCNN lacking iterations will be weak,so two strategies were adopted in order to keep its strong image segmentation capability:Special input image mentioned above was chosen to enhance the global coupled modulationfunction; The feature combination of "iterative local localized interactions" in STB/Itti wasadopted to assist the PCNN in performing the function of pulse synchronization.Experimental results show that the integration model of STB/Itti+PCNN can efficientlysegment images semantically, with mean AUC value increased by127.94%; there issignificant difference at above0.99probability level between STB/Itti+PCNN model andSTB/Itti model, and STB/Itti+PCNN model is robust against noises and geometric attacks.(3) To evaluate the semantic image segmentation results scientifically, eight criteria ofthe best semantic image segmentation are proposed as follows based on the results of thisresearch and some related literatures.1) Based on biological visual mechanism to some extent;2) Does not require any training, trials or tunable parameters;3) Emphasize on the largestsalient object;4) Uniformly highlight the whole salient regions;5) Establish well-definedboundaries of salient objects;6) Disregard high frequencies arising from noise and geometricattacks;7) Efficiently output full resolution image segmentation results;8) Can beimplemented by hardware easily. Meanwhile, we use these eight criteria to guide the design,building and implementation of the semantic image segmentation models in our research.(4) To obtain the best semantic image segmentation results, the GBVS+PCNN model isproposed according to the theory from coarse segmentation to fine segmentation. Afterresearching on and comparing with visual effects, performance indices and averagetime-consuming of nine current SVAMs, the GBVS model was chosen to implement thecoarse segmentation, and the intensity feature map extracted by GBVS was used as the inputimage of PCNN which extended GBVS to implement the fine segmentation; Finally, a salientregion identified algorithm based on "AUC Value" was proposed for automatic output of thefinal semantic image segmentation results. Experimental results show that the GBVS+PCNNmodel can meet seven of the eight criteria of the best semantic image segmentation, while thePCNN part of this integration model can conform to all of the eight criteria. By using pairedStudent's t-test method, we have got the probability level of significant difference betweenGBVS+PCNN model and GBVS model, standing at above0.99.(5) Saliency map generated by PQFT model cannot locate the salient object of an imagewell, because there is too much redundant low-frequency information. To mimic the center excited-surrounding inhibited mechanism of biological visual neurons, a simplified C-Salgorithm is proposed based on the surrounding region identified by the linking coefficient setautomatically by PCNN. Three subtraction maps of CIE Lab color space channels calculatedby the C-S algorithm were taken as the coefficients of imaginary part of PQFT quaternionimage, through which the improved PQFT, named IPQFT, is proposed. IPQFT could greatlyreduce the redundant low-frequency information in its saliency map. Additionaly, to solve theproblem that the algorithm for automatically identifying the salient region based on "AUCValue" costs5%of average time-consuming in its integration model, a similar algorithmbased on "Size Change" is proposed, which makes the average time-consuming decreasedfrom102.0ms to only16.1ms in MATLAB environment.(6) To meet the practical application requirements of real-time semantic imagesegmentation method which can be implemented by hardware easily, the integration model ofIPQFT+PCNN is proposed. Just like GBVS+PCNN model, IPQFT was applied for coarsesegmentation; the PCNN extended IPQFT and was used for fine segmentation; the algorithmfor identifying the salient region based on "Size Change" was applied to output automaticallythe final semantic image segmentation results. Experimental results show that the averagetime consumed by IPQFT+PCNN model to process a testing image is about238.2ms inMATLAB environment, which achieves the real-time requirement; because the Fouriertransform and PCNN algorithms of the integration model can all be implemented by hardware,then the IPQFT+PCNN model can also be implemented by integrated circuit easily. Moreover,the IPQFT+PCNN model is robust to noises and geometric attacks, and it is parallel,automatic, and intelligent as well.(7) Comprehensive evaluation on performance indices of semantic image segmentationmodels or methods is still not perfect. According to the eight criteria of the best semanticimage segmentation, a total score of decision table is proposed to evaluate different kinds ofimage segmentation models or methods, which enriches the current evaluation methods orindices of SVAMs. According to the total scores of the decision table, the three integrationmodels proposed by this dissertation outperform the other nine current SVAMs in semanticimage segmentation, and can output semantic image segmentation results more accurately.