Human Visual Perception Motivated Research And Application On Salient Structures In Pedestrian Images

When humans interact with the outside world,80% of the information comes from vision. In the human visual system, the primary visual cortex receives information from the outside world via the eyes. The information is handled by low-level, middle-level and high-level visual processes. Points and edges, which can be grouped into middle-level visual features such as boundaries or regions, are two basic low-level visual features of an image. Based on middle-level visual features and their structural information, we can find whole objects, which we hope to learn to fully parse in future work. Images containing pedestrians are one of the most popular and familiar kinds of natural images. Facing a novel pedestrian image, humans can extract the useful and salient structures, the points, edges, boundaries, regions and so on, which belong to the pedestrian, and discard the useless ones, quickly and easily. They can perform perfect pedestrian parsing based on these salient structures.How can computer simulate the human visual perception system and complete that work, which are easy for human but difficult for computer, is the main task of computer vision field. This dissertation takes the Physiology of human vision and Cognitive Psychology as references, models the process using the relative knowledge of mathematics, parses the pedestrian based on the detection and analysis of salient edges, salient boundaries and salient regions which are low-level and middle-level features. This parsing model is applied in high-level visual processes such as the segmentation pedestrian, the estimation the orientation of the pedestrian and the pedestrian detection rescoring. Salient point is defined as the point which belongs to the contour of the object or the boundary between different regions. Salient edge is defined as the edge which belongs to the contour of the object or the boundary between different regions. More salient points the edge contains, more salient the edge is. Salient boundaries are defined as the boundaries which compose the contour of the object. And Salient region is defined as the region belongs to the object or foreground.The main works in this dissertation can be concluded as three points:1) Low-level visual feature of salient edges detection in pedestrian images based on Gestalt Principles. This dissertation used an easy method which combines scale space edges and tensor voting for salient point detection in pedestrian images. Based on the Gestalt Principle of "good shape", the number of salient points the edge contains, the length of the edge, the smoothness of the edge and the stability of the edge can be used to measure the saliency of the edge. This dissertation proposes a salient edge detection method based on the energy and stability of the edge. However, if the texture of background or object is complex, it is difficult to get good result only based on the features of the edge itself. Hence, this dissertation proposes another salient edges detection method which based on low-level and middle-level visual features. Salient edges can be detected by computing the difference of color and texture of the regions beside the edge and analyzing the "closure" of these edges. Both of these methods are also used in other natural images which contain different kinds of objects. This method not only considers the feature of the edge itself, but also uses color and texture which can describe the character of region. These salient edges will be helpful for salient boundary detection and also save the storage.2) Middle-level visual feature of salient boundary detection in pedestrian images. Salient edges can give a clue for object matching, recognizing, etc., but it is so difficult for them to be used in object segmentation and parsing. Based on salient edges, this dissertation proposes a salient boundary detection method in pedestrian images. Candidate soft boundaries are generated by gPb-OWT-UCM hierarchical segmentation algorithm under different thresholds. The gPb value on the boundary, the number of salient edges the boundary contains, the length and smoothness of the boundary are used to measure the saliency of the boundary. Based on the Gestalt Principle of "proximity", the salient boundaries which may belong to the object can be detected. Compare the shape of salient boundaries and ground-truth boundaries to the shape of the pedestrian templates separately. These salient boundaries segment the image into different regions. The shape of the region will be a powerful clue for high-level visual feature analysis.3) Pedestrian parsing based on the shape of the salient regions and its application in high-level visual processes. Different from semantic parsing, this model uses syntactic parsing to divide the pedestrian into several visual parts which can be observed directly in the image. These parts can be divided into perpetual parts and possible parts. The entity of perceptual tells us that the possible parts can be perceived by perpetual parts. Candidate segments can be generated from gPb-OWT-UCM hierarchical segmentation at different thresholds. Concentrate on the candidate segments, score them using shape feature, save the ones with higher scores, in other words, salient ones, and discard the ones with lower scores. Based on a pedestrian parsing tree, a bottom-up parsing process is used to parse the whole object. During this process, shape is still the main feature to be used. This parsing model also works well in high-level visual processes such as pedestrian and its parts segmentation, the orientation of the pedestrian prediction and pedestrian detection rescoring. All the results of these experiments verify the effectiveness of this model.