Research And Implementation Of Multimodal Information Fusion Annotation System For Image-Text Mixed Data

In the era of digital intelligent network,a large number of cultural digital resources are gathered,and new means and methods are urgently needed to organize and manage cultural resources effectively and rationally.At present,a large number of professional image-text mixed data have been accumulated in the field of culture,which is characterized by the mutual association of image and text,which is of great significance to the automatic annotation of image.This paper focuses on the humanities and arts books in the field of culture,and uses decorative images as the research carrier to implement the digital image-text processing and multimodal image annotation of image-text mixed data.The main contents of this paper include:(1)Aiming at the demand of multi-modal data for cultural big data,an adaptive image-text separation algorithm based on layout analysis is proposed.Using the professional image-text mixed books in the cultural field as the data source,the image-text information pairs consisting of images,titles and descriptive text are structured to form a multimodal data specimen database.(2)A new word discovery algorithm for domain-oriented lexicon construction is proposed.The algorithm improves the new word discovery algorithm based on information entropy and mutual information by combining a variety of statistical features with the text of professional books in the field of culture,completes the expansion and update of the domain lexicon,improves the low segmentation accuracy of the general word segmentation tool for professional words in the field of culture,and lays the foundation for the image annotation algorithm.(3)A multi-mode information fusion algorithm for image annotation is proposed based on the image-text separation algorithm,new word discovery algorithm and domain lexicon.The algorithm adopts PageRank-based multi-mode information decision fusion to merge the annotation information of both image and text modes,which ensures the richness and reliability of image annotation words to some extent.(4)Develop a annotation system of humanities and art books,and integrates the algorithm of image-text separation,new word discovery and multimodal information fusion image annotation proposed in this paper,which has certain practical value.This paper proposes a set of digital image-text processing methods for organizing cultural resources reasonably around the multimodal image-text datasets,and verifies the effectiveness of the proposed multimodal information fusion annotation method for image-text mixed data.