Ultrasonic ranging and infrared depth profiling for 3D image reconstruction and scene analysis

How to efficiently and precisely recognize and localize a target by generating a real-time 3D high definition image in a complex environment is a very challenging task. Traditionally, a target can be detected using ultrasonic ranging technique. However, the detailed information of the complex environment surrounding the target can not be detected by the ultrasonic ranging technique due to the low resolution of the depth profiling generated. In order to improve the quality of the depth profiling and acquired the detailed information, a robust method is required. Thus, in this study, a robust method has been proposed which combines the infrared sensor with the ultrasonic sensors to localize and identify the target in a complex environment by generating a real-time 3D high definition depth profiling and image reconstruction.;Our proposed method is mainly divided into two steps. First, the target is detected by using ultrasonic ranging algorithm. For this purpose, an ultrasonic array microphones system is developed which is called compact and programmable data acquisition node (CAPTAN) board. This system embedded with a Virtex-4 field-programmable gate array and 52 microphones to receive the ultrasonic signals. In this case, when the target is detected by the ultrasound signal sent from an ultrasound transducer, the reflected signal can be captured by the CAPTAN board, thus the direction of the target can be located and the distance of this target can be calculate simultaneously.;Next, detailed depth information of the whole complex environment is generated by the infrared sensors. A method called structured-light is used for generating the depth information. Kinect device is a good commercial product of infrared sensors which is developed by Microsoft with build-in infrared sensors. The infrared sensors are combined with the results from CAPTAN board to generate the detailed depth information. Finally, based on the detailed depth information, a real-time 3D high definition image reconstruction of the whole complex environment can be performed.;To reduce the processing time of 3D high definition image reconstruction, graphical processing unit (GPU) based acceleration is also integrated in the system. This GPU includes powerful compute unified device architecture (CUDA) for flexible and parallel processing. Parallel processing is implemented by writing the fused float-point multiply-add instructions. The parallel processing can significantly reduce rendering and reshaping time within the 3D image reconstruction procedure. According to the benchmarking results, the entire processing time for the real-time 3D high definition image reconstruction is reduced by 80%. The recognition and the localization of the target benefit a lot from the 3D high definition visualization of the complex environment.