A Hippocampal-entorhinal System Inspired Model For Visual Concept Representation And Understanding

Visual concept is the basic element of visual cognition.From the perspective of philosophy and cognitive psychology,a visual concept can be defined as an abstraction of a set of visual attributes.Understanding how the brain learns,organizes,expresses,and understands visual concepts is critical to building an artificial visual cognitive system.In recent years,the visual computing models based on deep learning have made remarkable achievements in the field of object classification,object detection,semantic segmentation and image description.Despite the great progress,these models do not care the way of visual concept representation and understanding in the brain.The traditional visual computing models tend to learn a mapping between language symbols and visual representations.And there are no explicit principles of visual concept organization and expression in these models.In addition,in the traditional models,there is no clear definition of visual concept understanding.In order to solve these problems,this paper proposes a visual concept space model(VCSM)to organize and express visual concepts.The VCSM is inspired by the generalized information processing mechanism of the hippocampus-entorhinal cortex system in the human and mammalian brains.In particular,this paper simulates the grid cells in the entorhinal cortex to construct a visual concept space.In the visual concept space,the visual concepts are represented by the ‘concept cells' which mimic the hippocampal place cells.As is well known,the humans' understanding of concepts involves two aspects: first,human beings can identify and reason about new concepts based on past experiences;second,humans can imagine corresponding examples based on the descriptions of a certain concept by others.Inspired by this,this article defines the visual concept understanding as concept inference(i.e.,inferring the underlying visual concept of the input image)and image reconstruction(i.e.,‘imagining' the corresponding image based on the given concept).Combining the visual concept representation and understanding,the contents of this paper are as follows:Firstly,the disentangled representation learning based on auto-encoders is studied thoroughly.The aim of disentangled representation learning is to obtain the disentangled representation of the input image(i.e.,using different informative dimensions of the latent variable to represent different generative factors,where the generative factors are assumed to specify the attributes of the image).Secondly,this paper proposes the attribute functions to represent visual attributes based on the disentangled representations.The attribute functions utilize a fuzzy information processing mechanism and ground the symbolic meaning of the visual attribute directly on the visual representations.Thirdly,this paper constructs the visual concept space by drawing inspirations from the information processing mechanism of the entorhinal grid cells.In the visual concept space,visual concepts are organized and represented by concept cells.The Concept cells are composed of attribute functions that mimic the characteristic of the hippocampal place cells.Finally,this paper investigates the ability of visual concept understanding of the model.The model needs to be able to infer the visual concepts underlying the input images,and can generate corresponding images based on the given concept(i.e.,with the capability of bidirectional generation).The paper applies the VCSM to the 3D Chairs and Extended Yale Face Database B data sets to validate its performance.The experimental results demonstrate that the VCSM can understand the visual concepts and generate diverse images.