Research On Problems In Image Retrieval Based On Multimedia Data

We study two types of user need:they not only want to quickly locate the wanted image(exploitation need)but also like to explore different related images(exploration need).Different image domains put different emphasis on these two types of need and we propose different structures to address such differences.To be specific,we study three typical domains:drawing image domain,landmark image domain and commodity image domain.We propose different structures to satisfy these two different types of user need.We study both unsupervised and supervised structure learning in methodology.In drawing image domain,we study content-based image retrieval scenario and focus on exploitation need from patent review.In landmark image domain,we study text-base query scenario and focus on balancing exploration and exploitation needs.In commodity image domain,we study recommendation scenario and focus on personalized need.In drawing image domain,we propose path structure to model lead curve.Lead curve connects drawing content to label text.It exists widely in drawing images,mixed with normal contours and deteriorating the quality of content-based image retrieval.We abstract the drawing image to a graph and transform lead curve detection problem to a problem of finding the path of the highest score.We customized supervised structured learning to detect the lead curve.The experiment result is good enough to match the real need.In landmark image domain,we propose tree structured summary to balance exploration and exploitation needs.We organize landmark images in a hierarchical summary,which offers users the flexibility to switch between exploitation need and explo-ration need.When we define hierarchical summary,we introduce the concept detail level to describe the detail extent of the image.Based on this concept,we design an objective function based on consistency criterion and diversity criterion.We exploit simulated annealing to search the best hierarchical summary.Both quantitative and qualitative experiments show that hierarchical summary satisfies exploitation need and exploration need simultaneously.In commodity image domain,we propose personalized list to satisfy user need.Based on collaborative filtering,we fuse image data and price data to improve recommendation performance.Experiment results show that fusing image data and price data improves recommendation significantly.At last,we study web image annotation,which is limited by domain.We propose to model annotation as a entirety by set structure.This transf'orms the image annotation problem to a problem of selecting the best subset of candidate tags.We design several joint feature representation to describe tags in each facet and the relation between tags across facets.Experiment results show that multi-facet annotation improves the performance.