Caption Text Detection And Extraction In Video

Nowadays the amount of digital videos increases explosively, and consequentlyit's a valuable research for computer to comprehend these multimedia documents,extract semantic information and boost some applications, such as video manage-ment, information retrieval and data mining. Caption texts embedded in videosare highly related to the video content and easier to extract than other semanticfeatures. They serve as a important clue for video content comprehension.We aim to design a prototype system and extract video text information.According to the proceeding sequence, it includes pretreatment, text detection,extraction and recognition. Our research focuses on the former three parts.1. Pretreatment includes video codec, image quality assessment and systeminitialization. In real application the system need to process videos with di?erentimage quality, and the proceeding method should be changed accordingly. In thispaper we propose a no-reference image quality assessment method. Firstly, ex-tract features from amplitude fall-o? curves and positional similarity on the imagefollowing natural scene statistics, and build feature vector accordingly. Secondly,train general regression neural network to predict image quality.2. Text detection is conducted to locate text boxes in frame images. Wepropose a fast and e?ective method. Firstly, calculate the edge image of a frameand revise it due to the phenomena of broken and conglutination. Secondly,label all CCs(connected components) on the edge image, filter out those frombackground partly, sort remained CCs by their position, search correspondingCCs and build text boxes depending on the geometric constraint. At last, conducttext box amalgamation to eliminate reduplicate detection results and text boxverification to eliminate false alarms.3. Text extraction is conducted to extract text strokes from text boxes.In text boxes, the color of characters can not be determined in advance andmany disturbances form background exist, so the procedure of text extraction isneeded. We propose a robust extraction method. Firstly, binarize the candidatetext box and conduct polarity estimation to determine on which polarity the textoccurs. Secondly, perform multi-frame verification and enhancement on text boxesemploying their temporal redundancy in video. At last, binarize candidate textboxes, execute CC filtering to remove disturbance from background, and generateclear binary image for recognition.We describe the experiment data set and results. They confirm the e?ective-ness and e?ciency of our methods.