Research On Content-based Indexing And Retrieval Methods For Large Scale Images

The boom of Internet and multimedia technologies leads to the explosion of multimedia data, especially images. Resource-abundant images bring much convenience. Howerver, there is an urgent problem needs to be solved. It lies in how to help people obtain their real interted images from huge amounts of images, especially those can’t be decribed clearly by traditial text. Under this background, we systematically do the research on the key technologies for indexing and retrieving content-based large scale image.In view of two problems of hard assignment:codeword uncertainty and codeword unplausity, a hyper-spherical soft assignment method is proposed. A hyper-sphere is constructed for each visual word which is the hyper-spherical center. The radius is denoted as the Euclidean distance from the visual word to the farthest feature in the corresponding cluster. A local feature in image is assigned to close visual words according to the relation between its location and the location of the each hyper-sphere associated to visual word. Hyper-spherical soft assignment is combined with the mehod of vector of locally aggregated descriptor for generating image decriptor. The experimental results demonstrate that hyper-spherical soft assignment can efficentyl increase the discrimination of image descriptor.To approximate image visual feature more accurately and reduce the memory requirement of storing image visual features, an enhanced residual vector quantization (ERVQ) is presented and used to construct multi-inverted index, thus the time efficicency on constructing inverted index is improved. ERVQ approximates the image features by training several codebooks. In the training procedure, a joint optimizing process is adopted so that the codebook of each quantizer is designed to minimize the overall distortion. In view of the problem of efficiency on searching exact nearest centroid in ERVQ, a lower-bound filtration based nearest neighbor search method is proposed. The high dimensional vectors are embedded into a low space where the lower bound of Euclidean distance between the vectors are computed for filtering the non-nearest centroids, so that the time costs on quantizing features are reduced. Besides, an ERVQ-based multi-inverted index is designed, thus, only a small number of centroids is needed for constructing an indexing structure which contains large number inverted lists. Experimental results show that, comparing to existing methods, ERVQ can significantly reduce the error of approximating image features and improve the retrival accuracy and efficiency, besides, the lower-bound filtration-based vector quantization can significantly improve the time efficiency of ERVQ.To improve the time efficiency when querying an image visual feature, two adaptive retrieval methods based on hype-sphrical filtration are proposed, which includes exhaustive filtration-based adaptive retrieval (EFAR) and non-exhaustive filtration-based adaptive retrieval (NEFAR). According to the location of query feature in the feature space, EFAR constructs a hyper-sphere whose center is the query feature, and the corresponding radius is calculated adaptively. Only the features that lie in the hyper-sphere are used for sorting, then the features dissimilarity to query feature are filtered. Based on this, NEFAR partitions each inverted list into several sub-clusters whose center is used to filter the features dissimilarity to query feature, so that the time costs on the filtration process are reduced. Experimental results demonstrate that both EFAR and NEFAR outperform existing methods over query time in the condition of the same retrival accuracy. Moreover, NEFAR is faster than EFAR.There are still many problems need to be studied in content-based large scale image index and retrival. For the image descriptor, how to combine the high-level semantic information with low-lever visual feature in images for generating image descriptor needs to further study, so that the content in image are represented better. For quantizing and encoding image visual features, a further sdudy is needed to projecting image visual features from high feature space into a low feature space and apply the ERVQ in the low feature space; meanwhile, the errors in quantization and projection are both considered. For the indexing structure, the designed inverted index is depend on the training dataset, so how to design a inverted index robust to real time updating needs further study.