Storage And Analysis Of Agricultural Digital Resources Under Big Data Environment

With the wide spread of Internet technology and rapid development the total amount of all types of data in society, we have already entered the era of big data. In the era of big data digital library faces two key problems: high-speed retrieval of digital resources,depth of mining and further analysis of these resources.The traditional relational database technology systems meet great challenges in the face of massive data retrieval and application performance. Therefore, in order to better integrate all types of digital resources, improve the application level of massive digital resources, provide higher level of knowledge services, library system should be innovative and timely introduct the Big Data technologies for new working situation.In this paper object of research is the data resources of the the National Agricultural Library.This paper analyzes the overall resources of the National Agricultural Library and the existing problems of current technical system. Compare Big Data technologies to current technical system in performance and function, and propose new technical framework for the digital resources in digital library, both in storage and applications based on big data technologies. This framework incorporates large data storage and processing technologies such as HDFS, Hbase and Spark. Then on the basis of the design of big data technologies framework build an experimental cluster with three nodes, and finished the construction of big data technology platform.The latter part of the paper discusses the the advantages of big data storage system based on Hbase and big data analysis system based on Spark.Use agricultural trade data as experiment for Storage and mining analysis and analysis the feasibility and technical capabilities of this new technology system. The agricultural trade data used in this study is stored in Hbase through HDFS distributed file System. And analyze performance of big data technology system and relational databases through the merits of experimental comparison. Finally, experimental results show that using the new technology system based on big data technology which this paper presents to retrieve data will be much more efficient than that int a traditional relational database System. Finally, by the advantage of high-performance computing and machine learning of Spark,use Spark GraphX which is new graphs and graph-parallel computation modle,conduct depth data mining of Agricultural trade data by using complex network algorithm.This study explore the using of digital resources by distributed storage and application in big data environment from many aspects such as data storage, data retrieval, data mining, etc,which has improved system performance compared to relational database technology and has certain practical significance...