The Design And Implementation Of Image Distributed Processing Platform Based On The Content

With the development of the traditional internet and the mobile internet, a number ofcontent providers such as microblogging, social networking are developing rapidly, on theinternet, the vast amounts of text, image and video need to be deal with in time every day,especially for image data, while a single server with limit processing capacity and noteasily scalable often become the bottleneck in the system as a whole.This article describes an image distributed processing platform for massive imagedata, finally provides a full range of distributed storage, computing, and retrieval services,and has a good system scalability and fault tolerance，the main features of the platforminclude content-based distributed image feature extraction and retrieval. As to imagefeature extraction, this paper mainly studies the content-based image feature extractionand matching, using SIFT features to describe an image. The local sensitive hash (LSH) isused to build the index, so that the similar image is likely into the same bucket and theretrieval speed is further accelerated. in distributed computing, we have realized a set ofimages distributed computing program based on Hadoop, completed efficient distributedfeature extraction and image matching. At the direction of distributed storage and retrieval,we can accommodate hundreds of millions of lines, hundreds of column table based onHBase, massive data storage service, while the design of the distributed index, to meet thefast distributed queries on the image.The experiments show that the image distributed processing platform for the massiveimage data have more efficient image computing power and reflects higher performance interms of storage and retrieval to avoid the shortcomings of stand-alone server processingspeed and the low scalability.