The Design And Implementation Of A CBIR System Based On Hadoop And Lucene

Recently, with the development of the Internet and multimedia technology, the global digital images are exploding. Tranditional text-based image retrieval technique has been not able to satisfy people’s needs. Therefore, to study and develop a content-based image retrieval system has become a hot field both in academia and industry.Hadoop is an open source distributed software under the Apache Software Foundation. It includes two main components of the HDFS and the MapReduce. In recent years, for its powerful ability of storage and processing of big data, Hadoop has been used in all fields of computing. In process of constructing a CBIR system, such as image feature extraction, feature data storage, image index creation and so on, data and computing quantity are very amazing. Therefore, to study how to base on Hadoop to build a distributed CBIR system will have important research value and broad application prospect.This paper introduces the design and implementation of a project of a CBIR system which author participated as an internship in the search group of Focus Technology Co., Ltd. The system can be divided into four modules:the storage system, the feature extraction system, the index system and the search system. The main characteristic of the project is to innovatively bring Hadoop into the construction of the CBIR system to overcome performance bottlenecks caused by big data. The construction of the index system and the search system also base on the inverted index and Lucene. The author was responsible for the overall design of the system, the implement of the feature extraction system and search system, and integration of all modules.The first two chapters of the paper introduce a survey of the development of CBIR systems and key techniques used in our system. The third chapter analyzes the requirement of the system and indroduces the design of the system according modules. In the section of the storage system, it introduces the design of the image database, image feature database and index database. In the section of the feature extraction system, it introduces the design of the feature extraction algorithms, the process the feature storage, and the static view and dynamic view of the system. In the section of the index system, it introduces the process of the index creation and static view of the system. In the section of the search system, it shows the static view and dynamic view of the system and simply introduces the design of the user interface. The forth chapter introduces the implement of the modules of the system in order of the third chapter with tables and codes. And, it introduces the construction of the test environment of the project and demonstrates the test results of the implemented prototype system. At the last chapter, it summarizes our works and looks forward to the future.