Researches About Visual Attention: Algorithm Design, And System Implementation

Attention selection is an important characteristic of human visual perception. Human being can easily perform general object detection/recognition task; however, traditional computer vision can not do well. In this paper, we focus on how to develop a human-like vision system in order to solve some basic problems in computer vision. The existing bottom-up attention selection models such as Neuromorphic Vision C++ Toolkit (NVT), SaliencyToolBox (STB) and etc. demand high computational complexity and can not work in real time as well. Some models with both bottom-up and top-down require human interactions as top-down signal and do not poessess online learning ability and visual memory. Based on these models, we propose an attention selection model with visual memory and online learning, a visual memory model with the amnesic function and a spatio-temporal saliency model that is fast enough to work in real time. The main innovations of this paper can be described into four aspects below:1. An attention selection model with visual memory and online learning is proposed, which has three parts: Sensory Mapping (SM), Cognitive Mapping (CM) and Motor Mapping (MM). CM is the novelty of our model which incorporates visual memory and online learning. Incremental Hierachical Discriminant Regression (IHDR) tree is introduced here to mimic visual memory. Self-Supervised Competition Neural Network (SSCNN) in CM has the characteristics of online learning since its connection weights can be updated in real time according to the change of environment. Eyeball Movement Prediction (EMP) mechanism is applied to estimate the movement of Foveation so that attention can be focused on interested objects. Our model was applied to implement object tracking and robot self-localization application.2. Most of state-of-the-art attention selection models used visual memory model without amnesic function. These models suffered from capacity overflow and low retrieval speed when dealing with continuous input image data. This paper proposed a novel visual memory model with amnesic function called Amnesic Incremental Hierachical Discriminant Regression (AIHDR) Tree. The tree could mimic human's Short-Term Memory (STM) and Long-Term Memory (LTM). Experimental results showed that our proposed model had a stable size, faster retrieval speed and higher accuracy when compared with non-Amnesic tree. This visual memory model was applied to the supervised video compression using visual attention. 3. Considering the high computational complexity of current saliency model, we propose a saptio-temporal saliency detection model based on Quaternion Fourier Transform (QFT). The value of each pixel in an image is represented as a quaternion composed of intensity, color and motion feature, and its phase spectrum is used to calculate the spatio-temporal saliency map. Experimental results show that our model can not only detect more objects in a video or an image than other models (NVT and STB), but also is fast enough to work in real time.4. We present a Hierarchical Selectivity (HS) framework based on PQFT to obtain the tree structure representation of an image by the multi-resolution characteristic of PQFT. With the help of HS, a Multi-resolution Wavelet Domain Foveation (MWDF) model is proposed to improve the coding efficiency in image and video compression. Experiment results on image and video compression illustrates that HS-MWDF based on PQFT can achieve higher image or video compression rate than Wavelet Domain Foveated Weighting (WDFW) model as well.