Research On The Framework And Key Technologies Of The Smart Ocean Big Data Platform

The rapid development of Internet of Things technology and computer technology has promoted the continuous development of smart ocean construction,and the ocean data is also expanding rapidly.This is both an opportunity and a challenge for the construction and development of a smart ocean.In order to make better use of massive ocean data,eliminate "information islands",enhance the level of ocean information,and promote the construction and development of smart oceans,this paper is based on the actual business needs of smart ocean construction,using a series of big data technologies to build a framework of smart ocean big data platform,research ocean data analysis model.Based on the framework of the smart ocean big data platform,the marine environment big data analysis and evaluation system is built to realize the application of big data technology in marine environment analysis and evaluation.This paper studies a series of key technologies of big data,and proposes a Hadoop-based smart ocean big data platform framework,including data crawling,aggregation,transmission module based on flume,data storage module based on hdfs and HBase,data processing module based on MapReduce,data migration module based on sqoop,business data storage module based on Mysql.And based on these advanced technologies,the framework of the smart ocean big data platform is built,and the processing flow of ocean data in the framework of big data platform is combed.At the same time,this paper proposes a marine environment data evaluation model based on brain storm optimization algorithm and BP neural network.The brain storm optimization algorithm is used to optimize the weight and threshold of BP neural network,and finally achieve the goal of accurately assessing the quality of marine environment.According to the business needs of the smart ocean,this paper divides the functional modules of smart ocean big data,and the overall design of the smart ocean big data platform prototype,and uses the advanced micro-service architecture design concept,combined with the spatial analysis technology and visualization technology of GIS.Based on the marine environmental data analysis model,the specific functions of the marine environment analysis and evaluation system are realized.The smart ocean big data platform framework and the big data based marine environment analysis and evaluation system can provide certain technical support for the future marine environment analysis and evaluation work,and can effectively utilize massive data resources,tap the potential value of ocean data,and promote the smart ocean construction is good and fast.