The Research And Implementation Of Distributed Big Data Retrieval Based On Custom Data Objects

Along with the advance of Internet+ age and the rapidly development of information technology and computer industries, the amount of data appears a trend of crazy increasing, which lends to a situation information flood and indicates the era of big data. However, great wealth value hides from view of big data, it becomes extremely difficult to scoop valuable information from the massive data in the age of data flood. What's more, in the information age, time equals money, people who grasp time means to size more opportunities than others and the chance of success will be enhanced, so how to search data and reduce searching time becomes extremely important. The technology of custom data objects distributed big data retrieval uses WebService technology to achieve the goals of quickly search in the large amounts of data information, it has a unique data definition specifications and design ideas, and uses the method of distributed technology. The system can retrieve the data only within the scope of the data object as long as the user specifies a data object, which greatly short the search time.Firstly, the article researches the current situation of distributed technology and data retrieval technology at home and board, and lists a variety of representative advanced technology in both areas, then, analysis the social significance about the research of our article. After that, introduces some related technologies used in the article, including Hadoop technology and the function of its various parts. The article introduces the WebService technology, such as its structure,working principle,the particular application in the issue and so on.Secondly, clearly puts forward the overall design ideas of the issue, and give a detail design for the input and output format, systematic introduces the principle and workflow in each section of the article, achieve the goal of specifically define for various of data objects in the issue. Then the article gives the key processes of the system some detailed description, including client working principle, servers working principle, node workflow specific process, etc. draws corresponding program flow chart and realize the function with C# language.The last but not least, compares the performance of custom data objects distributed big data retrieval system with traditional data retrieval system, the tests result shows that the subject system achieved the effect as good as expected, it has a lot of market value.