A Sensor-Fusion System to Detect, Track, and Identify People in Realistic Scenarios

For tomorrow's smart and assistive environments to fully realize, it will be of fundamental importance to be able to obtain the location of people in an environment, as well as their evolution in time---sometimes even across large sensing gaps. And so, the ultimate goal is to obtain a cheap, scalable solution for person-detection and tracking for use in long-term real-world scenarios.;In this work I present a system that takes a step in that direction. After a comprehensive review of existing human sensing approaches, I reason that the best multi-sensor configuration to solve this problem robustly and cost-effectively consists of cameras and inertial sensors. The proposed system localizes people using the existing infrastructure of CCTV cameras. People can, then, opt-in on being tracked and identified by carrying a mobile phone equipped with a custom software client. Using the inertial sensors on the phone, the client calculates and transmits a motion signature which is then compared with the motion hypotheses observed with the camera network. When a match is found, people are identified. The final solution is lightweight enough to potentially execute in real time on existing sensor nodes, thus providing a compact, cheap, and effective human-sensing solution. The system is evaluated through extensive simulations as well as a number of experiments.