Research On Real-time Anomaly Detection Method Of Inland Ship Trajectory Based On Spark

With the prosperity of inland navigation,how to ensure the safety of inland navigation becomes an urgent problem which should be solved.There are many marine traffic accidents every year,such as collision and grounding in inland waterways,which bring huge losses to the people’s lives and property.The real-time anomaly detection of ship’s trajectory can detect ship whose trajectory is abnormal and give warning timely.In this thesis,taking ship trajectory as research object,and studying from three aspects: fast clustering of ship sub-trajectories,real-time anomaly detection of ship trajectory and dynamic updating of the typical trajectory model.The main work and innovations are as follows:(1)A method of comprehensive similarity calculation between ship sub-trajectory is designed.Because the environment of river is complex,different sectors have different channel conditions.In this thesis,according to the characteristics of waterway,the research waters are divided into speed-limit area(such as confluence,ferry,bridge area and so on),width-limit area(wide / narrow waterway segment)and bend area.Then,calculating the similarity between ship sub-trajectories from the position,course and speed of ship sub-trajectory,and assigning different weights for different sectors to calculate the similarity of ship sub-trajectory more accurately.(2)A method of typical trajectory model of inland ships based on fast clustering on Spark plat is designed.Because the ship trajectory data has the features of high dimension,complexity and large quantities,a fast clustering algorithm for ship sub-trajectories is designed by improving the existing density peak fast clustering algorithm.Combining with the features of different sectors,the parallel clustering of the ship sub-trajectories is realized to improve the clustering efficiency based on the Spark plat.Then,typical ship trajectory model is constructed according to the clustering results.The feasibility and efficiency of the designed algorithm are verified by experiments at last.(3)A real-time anomaly detection method for ship trajectories based on Spark Streaming framework is designed.In order to meet the accuracy and timeliness requirements of ship trajectory anomaly detection,a real-time anomaly detection method for ship trajectory based on Spark Streaming framework is designed.In this thesis,three kinds of abnormal ship trajectories are defined: position-abnormal trajectory,course-abnormal trajectory and speed-abnormal trajectory.Firstly,detecting the position abnormality of ship trajectory points in real-time based on the constructed typical ship trajectory model.Considering the abnormality of single trajectory point’s course or speed is not enough to represent the abnormality of the ship trajectory,it’s supposed to take several position-normal points as trajectory section for the position-normal point;then replace trajectory point with created trajectory section to detect the course and speed abnormality of ship trajectory;and use sector’s threshold instead of global threshold to improve the accuracy of anomaly detection.Finally,the accuracy and timeliness of the designed algorithm are verified by experiments.(4)A dynamic updating method of typical ship trajectory model is studied.The ship’s normal trajectory will change due to the changes of inland navigation condition(such as dry season,wet season,etc.).In order to make the constructed typical ship trajectory model adapt to the dynamic changes of the normal trajectory,a online data stream clustering algorithm for ship trajectory is designed.The online clustering process is divided into two stages: incremental update of the local trajectory point micro-clusters and dynamic update of the global trajectory point micro-clusters.Then,updating the constructed typical ship trajectory model by using the global trajectory point micro-clusters,so that the model can adapt to the changes of ship’s normal trajectory to reduce the false alarm rate.Finally,the validity of the dynamic updating method is verified by experiments.