| Monitoring to detect unauthorized border crossing is very important for protecting national security. To accomplish this by continuous physical monitoring by border patrol agents is impractical. Networks of low-cost wireless sensors have been identified as a useful tool in monitoring with minimal human intervention. However, ensuring the effectiveness of unattended monitoring against an intelligent intruder is difficult, since the intruder can probe the system for weaknesses. To better understand the capabilities of such an intruder, we propose a model for an intelligent intruder whose purpose is to find a detection-free path across the border by which he could cross back and forth without the risk of detection. Our intruder model is composed of four agents, each with a specific task: Explore, Exploit, Evade, and Policy. The first three agents follow the best course for achieving their named goals. The Policy agent is an intelligent agent that learns on different maps whether to pick Explore or Exploit or Evade given the intruder's current knowledge about the map. We use Q-learning (QL) to train the Policy agent over various maps. In QL, the agent gets a positive reward for doing something right (e.g. moving to a detection-free zone closer to the goal) and a negative reward for doing something wrong (e.g. getting caught by a sentry). We investigated different factors that affect the intruder's behavior, like the effect of different rewards, different sensor coverage of the region, and faster and more effective sentries. Our results show that after getting trained on enough number maps, the agent becomes better at finding a detection-free path across the border region. In particular, it reduces the capture rate and the number of steps required to find a detection-free path. Therefore, an intelligent intruder like this can be used to build and test different defense mechanisms. |
