Independently Moving Objects Detection By Vision And Inertial Sensors

Moving objects detection with moving imaging platforms, also known as independently moving objects (IMOs) detection in the literature, has been the focus of computer vision research and is widely applied in areas such as industry automatization, video surveilence, artifical intelligence, medical image analysis and military applications. Independently moving objects detection which can be applied to pratical situations can not be easily achieved by vision sensor alone considering the complicated effects of platform motion, imperfect camera calibration and limited image resolution.Vision and inertial sensors are good candidates to be deployed together since each can be used to resolve the ambiguities in the estimated motion using the other modality alone. For instance, image measurements can help to counteract the error that accumulates when integrating inertial readings, and can be used to distinguish between the effects of sensor orientation, accerlation, gravity, and bias in accelerometer measurements. On the other hand , inertial data can help to resolve the ambiguities in motion estimation by a camera that sees a degenerate scene, such as one containing too few features, features infinitely far away; to remove the discontinuities in estimation motion that result from features entering or leaving the camera's field of view; to make motion estimation more robust to mistracked image features .This paper firstly presents a general introduction to the methods involving detection of moving objects by stationary imaging systems, moving imaging systems with vision sensors alone and by moving imaging systems fusing multiple sensors. Based on the analysis of inertial data and moving objects detection strategy by vision and inertial sensors, we have devoted our research to three particular sets of applications. (1)For the case of moving platform has neglectible motion components along the optical axis or varations of scene depth are relatively small in comparion to the absolute depth of observed scene, we proposed a novel IMOs detection method using monocular vision and inertial sensors by motion compensation. Experiments have been carried on the data collecting indoors using a monocular industrial camera and MEMS (Micro Electronic Magetic System) IMU (Inertial Measurement Unit) on a moving platform. (2) For the case of moving platform has neglectible rotation motion components, we proposed a novel IMOs detection method using monocular vision and inertial sensors on the basis of FOE (Focus of Expansion)-related optical vectors constraint. Experiments have been carried on the real-road mobile mapping monocular vision and geo-referenced image sequence of VISAT systems. (3) For the general case of a moving platform with no assumpation made on the motion characteristics or the observed scene, we proposed a novel IMOs detection method using binocular vision and inertial sensors under a spatial-temporal parallel resample particle filtering framework. Experiments have been carried on the real-road mobile mapping binocular vision and geo-referenced image sequence of VISAT systems. Various experiments of particular sets of applications have proved the effectiveness of the proposed methods.