Research On Key Technologies Of Network Moving Target Defense Based On Reinforcement Learning

Cyberspace has become the fifth largest domain of national sovereignty,highlighting the important strategic significance of "no cybersecurity,no national security".However,cybersecurity incidents have occurred frequently in recent years,which leads to severe security situation increasingly.Traditional network security mechanisms usually respond after cyber attack occurs,resulting in passive defense all the time.As an active defense technology that can change the asymmetrical advantage of attackers,moving target defense introduces the dynamics,diversity,and randomization into traditional static system,which aims to realize the dynamic changes of attack surface,and reverse the information advantage of attackers.Furthermore,network moving target defense belongs to the extension of moving target defense on the network layer.However,current network moving target defense methods still have following problems:(1)existing methods usually rely on rule-based strategy or attack-defense model.If actual network condition does not match assumptions,the defense strategy may be invalid,thus affecting the availability of network moving target defense;(2)current methods have serious impact on network performance,which are difficult to meet the demand of high-quality network services;(3)most methods adopt fixed mutation period.The inappropriate setting of mutation period may increase the network overheads(too short interval)or reduce the applicability of network moving target defense(too long interval);(4)various network moving target defense mechanisms consider specific network scenario,but pay less attention to mobile networks.Considering that current "cloudedge-end" network architecture has multiple network layers,network characteristics and security threats are different for each layer,this dissertation studies four network moving target defense mechanisms that consist of network address mutation and active service function chain migration on the "cloud" layer,network configuration mutation on the"edge" level and Ad hoc network route mutation on the "end" level.The main research content and innovation points include:(1)To solve the problems of inflexible defense strategies,timevarying network states,and insufficient survivability of network connections in current methods,a software-defined network address mutation method is proposed,and a seamless address mutation based on sliding window is designed to improve the survivability of network connection.A network address mutation algorithm based on "advantage actor-critic" is given,theoretical security analysis and experimental results indicate the effectiveness of our proposed method.In addition,a proof-ofconcept prototype system is implemented,and experiments are carried out with a variety of practical scanning tools.Compared with baseline algorithms,our proposed method can reduce the number of scanning hits by up to 25%with only a slight impact on network performance.(2)Under security threats including DDoS and co-resident attacks during the deployment of service function chains,an intelligence-driven active service function chain migration mechanism is proposed.Markov decision process is modeled to describe the dynamic arrival and departure of service function chains.By modeling the constrained satisfaction problem and removing infeasible migration actions in the action space,an adaptive proximal policy optimization algorithm based on the queuing model is designed to realize the attack-resistant migration decision and adaptive migration periods.Finally,the defense performance of our proposed method is evaluated by using multiple attack strategies and two real-world data sets called CICIDS-2017 and LYCOS-ISD2017 respectively.Experimental results indicate the superiority of our proposed method compared with benchmark algorithms.(3)Considering that existing network moving target defense methods are difficult to be applied in mobile networks,a network configuration mutation mechanism against DDoS in software-defined Internet of vehicles is proposed.The network configuration mutation of roadside units is modeled as Markov decision process,and deep reinforcement learning is adopted to solve the optimal network configuration.The trust assessment of vehicles is evaluated after switches,and spy vehicles are identified efficiently.Finally,experimental results verify the effectiveness of our proposed method.(4)Faced by packet drop attacks in vehicular Ad hoc networks,a cooperative Ad hoc network route mutation method is proposed,and a gridbased extended j oint action learning algorithm is designed to share learning parameters between vehicles,which aims to improve the convergence speed and avoid packet drop attacks.Finally,compared with representative algorithms,experimental results show that our proposed method has better defense performance.In this dissertation,we focus on the key technologies of network moving target defense,and conduct a comprehensive analysis and research from network address mutation,active service function chain migration,network configuration mutation and Ad hoc network route mutation.It mainly includes:model formulation,active defense strategy design,theoretical performance analysis,simulation experiment analysis,proofof-concept prototype system and prototype system evaluation.The results obtained in this dissertation are expected to have certain reference significance to the development of future cyberspace security and the construction of active defense system.