Object-based Representation For Scene Classification

How to encode and represent a scene remains a critical problem in both human and computer vision. Traditional local and global features are useful and have some successes; however, many observations on human scene perception seem to point to an object-based representation. In this paper, we propose two object-based high-level representations for scene categorization.In order to capture the object information contained in the scene, we utilize image annotations and semantic segmentation to construct our first object-based representation, and generate two corresponding scene classification model. The first object representation is consists of three main parts:firstly we calculate the object information directly to obtain object histogram; Secondly, we build spatial and geometrical priors for each object and each pair of co-occurrent objects from training scenes, and integrate the spatial and geometrical information of objects into the scene representation.Further, considering the unique advantages of the convolutional neural networks (CNNs) in object detection and recognition, we utilize convolutional neural networks to construct our second object-based representation, and generate a new scene classification model. There object information can be extracted from CNN pool5 layer.In the experiments, we verified the proposed three models respectively, and experimental results on public datasets demonstrate that the proposed two object-based representations are very powerful and discriminative.