Research On Failure-Inducing Input Generation For Cyber-Physical Systems Based On Deep Reinforcement Learning

Cyber-Physical Systems(CPSs)are composed of computational control logic and physical processes,which intertwine with each other.CPSs are widely used in various domains of daily life,including those safety-critical systems and infrastructures,such as medical monitoring,autonomous vehicles,and water treatment systems.Attacks on CPSs can have serious impacts that are difficult to estimate,so it is critical to test them effectively.However,it is not easy to obtain test cases that can be used to discover potential vulnerabilities in CPSs.In this work,we propose a failure-inducing input generation approach—FIGCPS,which requires no knowledge of the CPS under test or any history data of the CPS which are usually hard to obtain.Our method adopts the idea of deep reinforcement learning,interacts with the CPS under test,and efficiently searches for failure-inducing input guided by the rewards obtained from the interaction.Our approach is adaptive to collect information from CPS,which allows our method to reduce training time while maintaining the ability to explore different states of the system.In addition,our approach is the first to consider simultaneously generating failure-inducing input for CPS with continuous input space and CPS with high-dimensional discrete input space,which are common for certain classes of CPS.We perform experimental validation on three representative CPS simulators,and the experimental results show that our approach not only achieves a higher success rate than the existing state-of-the-art methods,but also explores two potential vulnerabilities in a well-tested CPS that are not explored by existing methods.