Object Detection And Distance Estimation Based On Multiplexed Images

Object detection is a fundamental and key task in computer vision;it has a wide range of applications in various commercial and military fields and is of great significance.Some applications,such as autonomous driving,not only need to detect positions of targets in the image but also need to estimate the depth of detected objects.How to detect and estimate the depth of objects efficiently has become a challenging problem.The multiplex imaging device uses a single imaging sensor to collect multiple channels of light simultaneously,reducing data communication and storage overhead.Multiplexed imaging devices are often equipped with image reconstruction algorithms,which restore images for each of the plurality of image channels to produce stereo images.Our work is based on the observation that both the appearance and disparity of every object are encoded implicitly in multiplexed images.Based on the existing object detection methods,we propose two detectors for multiplexed images.The proposed detectors can detect objects in the multiplexed image and estimate the depth of detected objects at the same time,at the cost of limited extra computation.We summarize our main contributions as follows:(1)A strategy,named "anchor pair",is proposed to predict multiple bounding boxes of each object in the multiplexed image.The multiplexed image encode multiple channels of light so that each object in it has multiple bounding boxes.The existing anchor strategy cannot predict multiple bounding boxes of each object well.To this end,this paper proposes the "anchor pair" strategy,which can associate all bounding boxes of each object in multiplexed images.(2)A one-stage object detection algorithm,named Disparity SSD,is proposed for detecting objects in multiplexed images.We combine the proposed "anchor pair" strategy with existing one-stage object detectors and propose a one-stage multiplexed images based detector.The experimental results show that the proposed detector can detect objects and estimate the depth of detected objects simultaneously in multiplexed images,with almost no extra inference speed overhead.(3)A two-stage object detection algorithm,named Disparity RCNN,is proposed for detecting objects in multiplexed images.We combine the proposed "anchor pair" strategy with existing two-stage object detectors and propose a two-stage multiplexed images based detector.Compared to existing detectors,the proposed detector can get a similar detection accuracy and inference speed.What's more,our proposed detector also estimates the depth of detected objects at the same time.Our proposed detector's performance in depth estimation is better than the fusion solution that combines existing depth estimation algorithms and object detection algorithms.