Research On Secure Control For Cyber-physical Systems Under Malicious Attacks

Cyber-physical systems(CPS)promote the development of communication,wireless network,distribution,artificial intelligence and other related technologies,and build the mutual mapping,timely interaction and high efficiency of human,machine,material,environment,information and other elements in physical world and cyber realm.CPS tightly integrates the computing,communication and control module,becoming the next generation of intelligent systems.As the core of intelligent manufacturing,the development of CPS has been valued by countries all over the world,such as the “American Competitiveness Program” and the EU's “ARTEMIS” project.Our party and government also attach great importance to the development of CPS.Both the report of the 19 th party congress and the 2018 Lianghui emphasized the need to promote the exchange and cooperation of production,learning,research,and application in the CPS artificial intelligence industry.Recently,cyber attacks occur frequently,causing great threats to national security,economic development,infrastructure security,and people's lives and properties.Therefore,how to ensure the security of CPS is an urgent problem to be solved.In order to promote the resolution of such problems,the National Natural Science Foundation of China has added the funding concerning security of CPS,and increased the establishment of scientific research projects related to CPS,and the Ministry of Science and Technology has also provided funding for national key research plans related to CPS security.Although the secure research of CPS has made great process,the proposed results on CPS security just relies on a single subject knowledge,such as control subject,computer science subject or network information security subject technology,the drawbacks of which have become increasingly prominent.It has been an urgent issue to investigate the secure control problems using multidisciplinary technical means.How to integrate multidisciplinary c to solve the security problems of CPS has become an important issue to be solved urgently.Focusing on the difficulties and key scientific issues in the research of CPS security issues,and combining control theory(sliding mode control,optimized control,etc.),machine learning(deep reinforcement learning),information theory(information entropy),and game theory(zero-sum game),network information security(moving target defense)and other multi-disciplinary technical means,this thesis investigates how to deal with malicious network attacks,design intelligent security control algorithms to ensure the security of CPS.The main contents,and approaches of this thesis are presented as follows:1.Chapter 2 investigates the problem of the resilient control for CPS under malicious sensor denial-of-service(Do S)attacks,which result in the loss of state information.The concepts of Do S frequency and Do S duration are introduced to describe the Do S attacks.According to the attack situation,that is,whether the attack is successfully implemented or not,the original physical system is rewritten as a switched version.A resilient sliding mode control scheme is designed to guarantee that the physical process is exponentially stable,which is a foundation of the main results.A zero-sum game is employed to establish an effective mixed defense mechanism.Furthermore,a defense-based resilient sliding mode control scheme is proposed and the desired control performance is achieved.Compared with the existing results,the differences mainly lie in two aspects,that is,one is that a switched model is obtained,based on which the average dwell-time like approach is utilized to derive the resilient control scheme,and the other is to employ the zero-sum game to make the attacks satisfy the concepts of Do S frequency and duration.2.Chapter 3 investigates the problem of optimal tracking control for CPS when the cyber realm is attacked by Do S attacks which can prevent the control signal transmitting to the actuator.Attention is focused on how to design the optimal tracking control scheme without using the system dynamics and analyze the impact of Do S attacks on tracking performance.First,a Riccati equation for the augmented system including the system model and the reference model is derived under the framework of dynamic programming.The existence and uniqueness of its solution are proved.Second,the impact of the successful Do S attack probability on tracking performance is analyzed.A critical value of the probability is given,beyond which the solution to the Riccati equation cannot converge.The tracking controller cannot be designed.Third,reinforcement learning is introduced to design the optimal tracking control schemes,in which the system dynamics are not necessary to be known.3.Chapter 4 investigates the zero-sum game based secure control problem for CPS under the actuator false data injection attacks.The physical process is described as a linear time-invariant discrete-time model.Both the process noise and the measurement noise are addressed in the design process.An optimal Kalman filter is given to estimate the system states.The adversary and the defender are modeled as two players.Under the zero-sum game framework,an optimal infinitehorizon quadratic cost function is defined.Employing the dynamic programming approach,the optimal defending policy and the attack policy are derived.The convergence of the cost function is proved.Moreover,the critical attack probability is derived,beyond which the cost cannot be bounded.4.Chapter 5 investigates the secure control problem for CPS when the malicious data is injected into the cyber realm which is directly connecting to the actuators.Based on moving target defense and reinforcement learning,we propose a novel proactive and reactive defense control scheme.First,the system(A,B)is modeled as a switching system consisting of several controllable pairs(A,Bl)to facilitate the construction of the moving target defense control scheme.The controllable pairs(A,Bl)can be altered to stabilize the system under certain unpredictable switching probabilities for each subsystem,which can prevent the adversaries from effective attacks.Second,both attack detection and isolation schemes are designed to accurately locate and exclude the compromised actuators from a switching sequence.Third,a reinforcement learning algorithm based on the zero-sum game theory is proposed to design the defense control scheme when there exist no controllable subsystems to switch.5.Chapter 6 investigates the deep reinforcement learning based secure control problem for CPS under false data injection attacks.We describe the CPS under attacks as a Markov decision process,based on which the secure control for CPS under attacks is formulated as an action policy learning using data.Rendering the soft actor-critic learning algorithm,a Lyapunov-based soft actor-critic learning algorithm is proposed to offline train a secure policy for CPS under attacks.Different from the existing results,not only the convergence of the learning algorithm but the stability of the system using the learned policy is proved.Also,only data is used in the proposed algorithm,and the algorithm is robust to system uncertainties and external disturbance.