Study And Implementation On Multimodal Commodity Searching For Online Shopping

With the development of Internet, electronic commerce technology is becoming mature, and lots of shopping website platforms have emerged. Online shopping has already been an important way for shopping. When we go shopping online, we firstly need to search what we want according to the information of commodities. However, nowadays, the searching engines of most websites are based on text. If customers have no idea with how to describe a commodity accurately, it is difficult for them to do searching on the websites. Consequently, the multimodal commodity information search technology has been studied in this thesis, which is based on the combination of text and image as input for online shopping.Firstly, the text features have been extracted using text segmentation and lexicon building in this thesis, and the image features have been extracted respectively from global and local directions. Then different preprocessings have been made based on different features, thereinto, the images have been classified using the opening software, weka, with contour features, and SURF points have been filtered based on affine transformation invariability of SURF feature. The experimental results show the superiority of the proposed method of feature extraction and preprocessing on the commodity image classification using the taobao dataset.Secondly, the indexing structure has been builded using Lucene after feature extraction and preprocessing. In this thesis, we have implemented four kinds of searchings based on the partition of text information structure, which are commodity name, trading capacity, commodity description, and commodity price. The experimental results show that the method of text searching proposed in this thesis is effective for these four kinds of searchings on the taobao dataset.Thirdly, the improved LSH algorithm has been realized for image searching after feature extraction and preprocessing. After analyzing the original LSH algorithm, the improvement has been done from two aspects as following:For the global features, Count PCAp-LSH algorithm has been used not only for reflecting the distribution of the data, but also for choosing the better projection axes with the higher suitability; For the local features, the p-LSH based on axial points has been used for improving the retrieval efficiency and accuracy, and the improved p-LSH has been proposed for the local features with high-dimensionality and complexity. The experimental results show the advantage of the proposed method on search quality, by comparing the searching results with the different parameters, the different characteristics and different search algorithms on the taobao dataset.At last, to compliment the multimodal searching engine, text searching and image searching have been combined in this thesis. The experimental results show the proposed methods in this thesis can satisfy different search requirements of users.