Social Multimedia Analysis And Summarization

Multimedia content analysis is a classical research problem in computer science.Traditional multimedia analysis and summarization works focus on pure content analy-sis,wheredataqualityishigh,andcomesfromprofessionalwebsites. Socialmediamakesmultimedia data generated by users become the primary source of data, which is large-scale, of low quality, and requires social and personalized features. Users are hard toefficiently extract interesting information from huge amount of data. Therefore, it is sig-nificant to develop automatic social multimedia analysis and summarization techniques.With the opportunities of social multimedia, there are also some challenges. First,user generated content makes multimedia increase its amount in social networks. How toeffectively organize and represent multimedia information is of great challenge. Second,information production and propagation become sparse, which makes it difficult to dis-cover popular topics from global aspects. Third, people are used to short messages’ fastconsuming, which makes longer posts more likely to be ignored. How to find personal-ized and interesting message from a huge amount of data is challenging for every Internetuser. To solve the above challenges, this thesis proposes works accordingly. The maincontent of this thesis includes:1. Extensive analysisof featureextractionalgorithmsandtheir fusionalgorithm. Fea-tureextractionisthebasicofanyimageprocessingwork. Tosolvedataorganizationand representation problem, this thesis chooses representative feature extraction al-gorithmsforanalysis. Theirperformancesarecomparedonlarge-scaledatasets,andprinciples of feature extraction and fusion are summarized, which provides motiva-tions for other applications when selecting features. This thesis further proposes afusion algorithm to take the advantages of each feature. Experiments show that thefusion algorithm performs better than individual algorithms.2. Flexible bilateral correspondence model for image and text on social media. Socialmultimedia is complex with various correspondences. To solve hot topic discoveryproblem, this thesis proposes a bilateral correspondence Latent Dirichlet Alloca-tion model to cluster and summarize multimodal microblog data. This model canflexibly fit the social media data, and experiment results proved this. 3. Social-sensedvideostoryboardingalgorithmforvideosonsocialmedia. Comparedwith traditional movie or sport video summarization, social video has various typesof content and new requirement of user interests. To solve the personalized videosummary problem, this thesis proposes a novel social-sensed personalized videostoryboard algorithm. This algorithm not only considers content importance of avideo, but also pays attention to user interests.