Research On Visual Urban Attribute Perception Methods

Visual urban attribute perception has received a lot of attention for its importance in many fields.It has long been recognized that urban residents' perception on the city has a certain degree of influence on the quality of education in the city,the health of urban residents,and the social activities of people.Thus,visual urban attribution perception has become a hot research topic in computer vision and multimedia field.Most of the current research methods predict the perceptual degree of an image on a perceptual attribute directly.However,the visual perception of an image is a subj ective impression of human,it is really difficult to make an absolute prediction for an image.Therefore,we transform the visual urban attribute perception task from a simple regression or classification task to an image ranking task,and achieve perceptual prediction by ordering the degree of perception for an image on the perceptual attribute.Due to the subjectivity of human perception,if we predict the perceptual attribute or the degree of perception for an image on the perceptual attribute directly,its accuracy will often be affected.In this paper,we propose a method named visual urban attribute perception with deep semantic-aware network to transform the visual urban attribute perception task into a ranking task by pairwise comparison of images.And we use deep neural networks to predict the specific perceptual score of each image.Distinguished from existing researches,we highlight the important role of object semantic information in visual urban perception through the attribute activation maps of images.Base on this concept,we combine the object semantic information with the generic features of images in our method.In addition,we use the visualization techniques to obtain the correlations between objects and visual perception attributes from the well trained neural network,which farther proves the correctness of our conjecture.The experimental results on large-scale dataset validate the effectiveness of our method.We noticed an important fact ofthe visual urban attribute perception:we need a large amount of data with labels on this task,and only the perceptual attributes with a large number of artificially labeled samples can be predicted.At the same time,the perception of human on images contains various attributes,and the difficulty of data annotation determines that we cannot prepare enough annotation data for all perceptual attributes.The conflict between them is a major difficulty to solve the visual urban attribute perception task.Inspired by the visual urban attribute perception with deep semantic-aware network,we find that there is a certain semantic correlation between the perceptual attributes.And we propose our new work,visual urban attribute perception with zero-shot learning to predict the perceptual scores for new perceptual attributes which have no training samples according to perceptual attributes with training data.We train the networks of the perceptual attributes with labeled data,and then use visualization of the network to mine the relationships between the perceptual attributes and the semantic information in the images,so as to obtain the feature representation of each perceptual attribute.For the new perceptual attributes,we get the representations using the correlations between perceptual attributes,or the test sample,we project images into the feature space where the perceptual attribute feature is located,thus realizing the perceptual prediction of images on the new perceptual attribute.Comparative experiments with other zero-shot learning methods demonstrate the superiority of our proposed method.