Study On Shape Feature With Deformation Robustness And Its Application In Retrieval

This thesis focuses on shape descriptors, and their application in Content-Based Product Image Retrieval (CBPIR). Shape is an important visual feature, and it plays a key role in object recognition. Robustness to deformation is of crucial importance for shape descriptors due to noises and distortions in images. Content-Based Image Retrieval (CBIR) is a classic topic in pattern recognition and computer vision. With the prevalence of E-Commerce sites such as Ama-zon, eBay and Taobao, searching product images on these websites has become an emerging application, which makes CBPIR a promising application-oriented research field.In traditional CBIR, generally the proposed algorithms are verified on im-age datasets that are collected from image hosting websites such as Flickr and social networking sites such as Facebook, and scene images take a large per-centage of these image collections. Most state-of-the-art methods utilized lo-cal features such as SIFT and SURF, with Bag-of-Words model and Spatial Pyramid Matching. But these methods donot work well in CBPIR. The main reason is that in product images the detected feature points are much less than that in scene images. However, some advantages makes CBPIR good scenario for employing shape features.1) Product images have simple backgrounds, which make image segmentation easier.2) The objects (products) stand out in product images, and this give us important prior information for object (prod-uct) detection.3) Most products have appearance stableness:objects in images have relatively stable forms and appearances. Firstly, many products are rigid bodies. Secondly, people tend to take product photos from certain angles. For examples, most T-shirt images are in spread form.Shape extraction and shape representation are two key problems in apply-ing shape features. In shape extraction, we proposed a main region extraction algorithm based on connection areas in product image that have relatively sim-ple background. Later we proposed a generic object detection and segmenta-tion algorithm based on image segments. This algorithm can apply on images containing salient objects, including product images with more complex back-ground. In shape representation, we focus on Shape Context (SC) descriptor. We apply fuzzy model to Shape Context and improve its robustness to shape de-formation. We also introduce ordered Bag-of-Words model to construct novel descriptor that can applying real-world applications.In this thesis, we introduce a mobile product image retrieval system. We verify the effectiveness of shape features in CBPIR. The query image is taken by mobile devices and uploaded to the server. The users are asked to take the im-age following certain instructions, so that we can have prior information about the location and size of the product in images. Our algorithm can extract main object region based on connected areas in images. This algorithm applies on images with simple background, which are classified by a Support Vector Ma-chine (SVM) classifier.Compared with specified object detection models and approaches such as pedestrian detection, vehicle detection and face detection, generic object detec-tion is not so well-studied. In specified object detection models and approaches, the researchers need to build models of the targets and detection them in im-ages. But in generic object detection and approaches, we need to deal with a more abstract conception, the "object". We need to figure out on basic ques-tion:What makes an object, which is more difficult. We extract and analyze cues either describe the dissimilarity between a region and its surrounding area, or describe the properties of the region itself. We integrate these cues under a Bayesian model and produce a score to measure how likely the region is to be an object.Shape Context is a widely used shape descriptor. Although the idea of fuzzification has introduced into Shape Context research, there is no well-defined fuzzy Shape Context. We propose a fuzzy Shape Context, and further construct multi-level fuzzy Shape Context and multi-scale fuzzy Shape Context. There new descriptors are more robust than the original Shape Context.A crucial constraint of applying Shape Context in real-world applications is its slow matching. Solutions aiming to this problem follow either Bag-of-Words model or Dynamic Programming. We incorporate these two ideas. In Bag-of-Words model, since spatial information is totally lost after summariz-ing the quantized feature points as a histogram, the generated histogram of Shapemes suffers a performance degradation. We introduce ordered Bag-of-Words model to improve the performance of the descriptor by keeping the order information, and utilize Dynamic Programming in sub-histograms matching for fast matching.