Design Of A Big Data Platform For Agricultural Information In Orchards

With the rapid development of agricultural information technology,more and more orchards in China are using agricultural information collection systems,irrigation control systems and expert systems based on wireless sensor networks.These systems are constantly generating large amounts of data,and as this data grows,it becomes increasingly difficult for traditional relational databases to meet the demands of storing this data.How to store these data efficiently and find valuable information to better serve agricultural production has become a new topic of agricultural informationization.In this paper,the storage requirements of agricultural information data in orchards are analyzed in detail,and a Hadoop-based big data platform for agricultural information in orchards is designed,based on the agricultural information data from several orchards in Guangxi and Huizhou.The platform is mainly composed of two parts: data collection system,which is mainly responsible for the collection and transmission of orchard agricultural information data;data storage system,which is mainly responsible for the storage of orchard environmental data,video data and network crawl files.After completing the design of the big data platform,the Hadoop cluster was deployed and various components of the Hadoop ecosystem were debugged.Finally,based on the crawled files and agricultural information data from orchards in Guangxi and other places,the data query and small file storage experiments were carried out on the Hadoop cluster,and the My SQL+HBase based agricultural information data storage method was proposed,which improved the data query speed of more than twice the million level without reducing the small data volume query.It also improves the original small data storage method of the Hadoop ecosystem,and designs a small file multi-level processing framework based on the greedy algorithm,which can effectively reduce the memory consumption of the main node of small file metadata information over 80%.The orchard agricultural information big data platform can realize the storage,disaster recovery and reuse of the subject orchard agricultural information data,and the performance of reading and writing is significantly better than the traditional orchard data storage platform.Therefore,this paper is based on Hadoop’s design and development of orchard farm information big data platform has important practical significance to promote the development of agricultural information.