Research Of Temporal Spatial Fusion VSA Model And Its Application In MPEG Compressed Domain

With the development of biological science, many research findings show that human eyes'attention to a visual image varies from region to region. According to the principle of human visual, "attention" is an important psychological adjustment mechanism during the human processing information, it can distribute the limited information processing resources which make the perception have the ability to choose. This selective and initiative psychological activity is called the visual selective attention (VSA). Visual selective attention is the key problem in researching human visual system with the significance of the evolution theory it guarantees that individual can locate one's limited mental resources on processing the stimulation or events which is vital for individual's survive in limited time. Researchers in different fields have already done numerous works in selective attention mechanism and have put forward a series of theory and model. The calculation model research of visual selective attention is aim to building the different forms of algorithm and process of the modeling and application on the basis of a thorough study on the choice of human visual attention the mechanism. In the visual selective attention model field, the most classical VSA calculation model is proposed by Itti.In this paper, I do in-depth analysis and research of the Itti VSA model, at the same time, points out the shortcomings of this model. Itti VSA model mainly aimed at analyzing the static images and static saliency characteristics but lack of the extraction of saliency characteristics of a dynamic object, and therefore cannot deal with the image of dynamic sequence—video. According to these shortages, this paper has made some improvement which is the establishment of a new visual selective attention model called Temporal Spatial Fusion VSA model(TSF). First of all, according to the characteristics of the human vision, the saliency features are divided into two categories: static and dynamic. Then, through light flow calculation and global motion estimation and compensation, real information about the movement of the object could be concluded. On this basis, dynamic saliency characteristics are extracted from motion amplitude and motion consistency these two aspects. On the generation of the whole saliency map, this model is different with Itti VSA model. This model adopts adaptive dynamic weights fusion way which is more effectively achieve the fusion of static and dynamic significant characteristics.On the basis of a thorough study on MPEG video coding standard, the TSF VSA model is applied to MPEG compression domain video saliency region extraction Through the analysis of MPEG video, DCT coefficients and the MV these two kinds of important information are extracted without any undecoded. Then, the DCT coefficients are reconstructed to extracte four kinds of MPEG compression domain static saliency characteristics which can reflect the brightness, color, direction and texture respectively, then MVs are pretreated and extracted motion amplitude and motion consistency these two MPEG compression domain dynamic saliency characteristics. After extracting the static and dynamic saliency characteristics successfully, it can generate overall saliency map and detect the MPEG compression domain video saliency region assisted with the TSF VSA model. The experiment results show that this method can extract saliency region effectively and the computational complexity and real-time reached the expected effect.