Three dimensional localization and tracking for site safety using fusion of computer vision and RFID

We propose a state of the art fusion framework of Radio Frequency Identification (RFID) and Computer Vision (CV) to support object recognition and tracking in a three dimensional space. Fusion can significantly improve performance in applications of autonomous vision and navigation and site monitoring, especially in outdoor environments. Increasing safety in construction zones and enhancing security in airports are important problems that involve understanding interactions between objects, machines and material and can be solved using sensor fusion and activity analysis. Identifying objects solely via vision is computationally costly, error prone, limited by occlusion, and sometimes impossible in practice. RFID can reliably identify tagged objects and can even localize targets at coarse spatial resolution. Alternatively, CV can increase the performance of RFID by fine tuning the location information and providing fuzzy features to avoid cloning or deception. Therefore, RFID and CV provide both overlapping and unique information for deciding on object ID, location, and motion.;We have implemented stereo processing using commodity cameras and have used a commercial RFID based Real Time Location System (RTLS) for our experiments and have achieved encouraging results. The performance of both modalities was evaluated separately and in fused mode. In our stereo experiments outdoors we obtained an RMS accuracy of within ∼7.6 in ( 19.3 cm) for objects up to 80 ft (24.4 m ) away from the cameras. For real time trajectories, RTLS provided 2 m to ∼2.6 m location accuracy for dynamic tagged objects in a cell of 40x40 m with four readers. We propose a fusion based tracking algorithm and our research demonstrates benefits obtained when most objects are cooperative and tagged. We abstract the information structures in order to support a Site Safety System (S-3) with diverse information sources and constraints and processes that may not have knowledge of each other. We have used relaxation to control the integration of information from CV, RFID, and naive physics in tracking. The label elimination approach readily represents the ambiguity occurring in real-life applications. The key to reducing the computational requirements is to eliminate many labels at each filtering step while keeping those labels compatible with observation. As a post processing step to labeling, we have used total track smoothness for optimization to update computed tracks for increasing system tracking reliability. Work site analysis can proceed even when information from one sensor or information source is unavailable at some time instances.;We have shown with simulations and real data that fusion can greatly increase tracking performance and can reduce computational cost and combination search space up to 99% in some cases. Test cases showed how fusion can solve some difficult tracking problems outdoors. We assessed performance of tracking using track error i.e fraction of wrong trajectory point assignments. For some object trajectories outdoors, the fused system reduced the track error from 0.53 to 0.13. The likelihood of producing correct object trajectories in regions partially or fully occluded to CV is also increased. We conclude that significant real-time decision-making should be possible if the S-3 system can integrate information effectively between the sensor level and activity understanding level. Engineering faster RFID updates will likely reduce the number of objects that can be sensed; however, this should be a favorable tradeoff in a construction site. Employing knowledge based constraints and analyzing systematically object track initiation and termination are some of the possible research expansions to be worked upon in the near future.