Research On Textual Saliency Analysis And Text Detection In Videos

With the popularity of digital products such as smart phones,and various online social media allowing the uploading of videos taken by individuals,such as Youtube and Facebook,have increased,video becomes an effective carrier of information trans-mission.To detect salient image regions containing textual patterns in the video is valuable to many content-based video applications such as video retrieval,abstraction,classification and analysis.However,the flexibility of image acquisition styles,vari-ation of text contents and complexity of environments make the textual extraction in videos a challenging task.At the same time,it has received extensive attention in many research fields such as pattern recognition,computer vision,image processing,multi-media technology and so on.As a key part of text information extraction in video,we studies the text saliency analysis and text detection in video,and proposes the corre-sponding effective algorithms.An effective text saliency analysis in videos integrating both spatial and tempo-ral textual features is proposed.We first compute text-alike confidence values of local image regions,which capture the basic visual cues of textual components in the video frames,using an efficient cascaded predication model.Next video frames are seg-mented into region of uniform size,and then we construct patch features depicting the statistical and spatial distribution of confidence values and combine them with general visual features like colors.Then we proposed a spatiotemporal textual saliency anal-ysis model based on random walk with restart on the fully connected directed graph of local video regions(depicted by Markov process),then we can compute the spatial saliency map based on textual confidence values and general visual features.On the other hand,the temporal stability of the text is calculated based on the stability of the text information in the continuous multi-frame video images.Finally,the random walk with restart algorithm is used to combine the spatial and temproal saliency map.A text detection method of videos exploiting the correlations between the text and its background region in the scene is presented in the thesis.We take the symbi-otic relationship of text and its background region into consideration and regard that as the core feature of filtering false candidates so as to enhance the textual detection performance.Specifically,we propose a seed localization and growing algorithm for localizing the foreground text components in the video context,and on the other hand,we extract local homogeneous regions in the scene as potential background candidates,and get the sign background region by setting text-hole validation and edge consistency constraints on it.We then employ a bipartite graph model with random walk algorithm by setting foreground text components and sign background regions as disjoint sets of nodes.This model captures the spatial,appearance and motional correlations be-tween the text and the background and filters out inappropriate text candidates,then the resulting text components are further aggregated into text lines.the effectiveness of proposed methods have been validated on several public s-tandard scene video datasets such as ICDAR2013,ICDAR2015 and so on.Experi-mental results show that,compared with the existing approaches,the proposed method of video text saliency analysis based on the fusion of spatiotemporal features and the video textual detection method combining text and background information effectively improve the accuracy,and achieve the desired goal.Meanwhile,the proposed methods have the potential to further improve in the follow-up work.