Design And Implementation Of Fiber Disturbance Event Detection Platform Based On Hadoop

Optical fiber has been widely used in communication systems due to its advantages of communication frequency bandwidth,low transmission loss and strong anti-interference ability.When fiber optic communication fails,it is important to be able to quickly locate fiber disturbances and classify disturbance events(including breakpoints,disturbances,and stresses).Long-term fiber-optic disturbance detection will generate massive data.Big data technology can achieve efficient storage and analysis of massive data.This thesis comes from the “Submarine Cable Testing” project.In this thesis,a Hadoopbased fiber optic disturbance event detection software platform is created.The platform uses hardware to collect fiber sensing signals,and uses network communication,database,Hadoop and other technologies to monitor fiber disturbances and manage big data.The main work of this paper is as follows:(1)According to the specific project requirements,after in-depth study of optical fiber sensor location technology,big data technology,WebService and random forest model,this thesis proposes the overall design scheme of Hadoop-based fiber optic disturbance event detection platform.The scheme consists of two parts: fiber optic disturbance monitoring software and data management system.The fiber optic disturbance monitoring software collects and stores data in real time and classifies and locates disturbance events,realizing real-time monitoring of fiber running status.The data management system uses Hadoop software library such as HDFS,HBase and MapReduce to realize data distributed storage and parallelization disturbance event classification.(2)The fiber optic disturbance monitoring software contains five functional modules.The communication protocol module formulates management instruction communication protocol and data acquisition communication protocol.The database module completes the design of the data table,and optimizes the installation,creation of the database and data storage.The management instruction communication module implements parameter configuration of laser and embedded device based on TCP.The data processing module implements data collection,analysis and processing based on UDP.The disturbance event detection and location module implements real-time classification and location of disturbance events by using correlation analysis and threshold setting.(3)The data management system is designed and deployed on a small Hadoop cluster built on the server side.The system collects optical fiber sensing signals distributed in various regions based on C/S and stores the data in MySQL;uses Alibaba's open source Canal framework to transfer data from MySQL to HBase;uses MapReduce and Mahout establishes a random forest model to realize the classification of parallelized disturbance events.A remote interface call service based on WebService is developed for users to query historical data and event classification results.(4)The monitoring software and data management system were tested and verified.The monitoring software performs performance and functional tests;the management system performs data collection,transfer,storage,disturbance event classification,and WebService service testing.The test results show that the Hadoop-based fiber optic disturbance event detection platform designed in this thesis can realize real-time detection and localization of fiber disturbance events,and the positioning accuracy and real-time performance meet the design requirements.The application of the Hadoop technology realizes the distributed processing of massive data,laying a foundation for further mining and commercialization of big data.