Personalized Annotation For Mobile Photos Based On User's Social Circle

As the amount of photos in user's mobile devices expands,how to organize,manage and index these personal photos has become a much more pressing issue for users due to the large data set size.Automatic image annotation is an effective andpromising technique to solve this problem.Different from traditional content-based photo annotation,for mobile photos annotation,users are more interested in the context information behind the photos.On the other hand,there are millions of users upload and share their personal photos in the social media platforms.These photos record the uploader's life.So,the user's social circle can provide valuable information for it.In this paper,we propose a personalized annotation framework for personal photo collections leveraging the user's social circle.However,the accompanying textual information of social network is sparse and ambiguous in nature.To address the unreliability problem of social network,we generate reliable tags for social photos before assigning tags to the user's unlabeled personal photos.Considering that event is one of the most important elements of people's life and memories.And most of the personal photos are taken during specific events.The same event should share the same event attribution labels.We detect social events at first to generate reliable event tags for social photos.In the tag generation stage,a multi-modality hierarchical clustering algorithm is performed to detect social events at first.Innovatively,we use “Album” instead of individual photo as the basic unit for clustering.Finally,inspired by similar images are more likely to have the same labels,we employ a weighted nearest neighbor model for label propagation,generating personalized tags for unlabeled photos in the user's mobile devices from the user's social circle.To validate our proposed framework,we evaluate our framework on two data sets,the existing public dataset ReSEED crawled from Flickr and a new,large-scale,real-world dataset collected from Renren,the largest Facebook-like social network in China.The evaluation results show promising results of our proposed framework.We also analysis the two datasets to verify our statements about social network.