Design And Implementation Of Face And Pedestrian Perception System Based On Depth Camera

Computer vision is an important research field of artificial intelligence.Object de-tection,basic task of computer vision,is the research focus of academia and industry.The machine perception about person is widely used,especially in the intelligent securi-ty,self-driving cars,mobile robots,etc.Depth cameras are used for realizing the percep-tion to achieve faster and more accurate 3D positioning.A part of commercial and open source codes products are only applicable to a specific camera or scenes,which can be neither secondary developed nor modified.This thesis proposes a face and pedestrian perception system based on depth camera,which can be easily developed and conve-niently extended.The system consists of four layers:hardware layer,driver layer,application layer and visualization layer.The functional units in each layer are independent of other units in the same layer or from other layers.All the functional units have the same unified format,which makes them easily called and extended in the form of plug-ins.The hard-ware layer is compatible with heterogeneous camera devices,including a variety of depth cameras and color cameras.The driver layer makes the camera interfaces unified.A unit of the application layer can be easily called by other units,for instance,the detection unit is called for service of the tracking unit.The visualization layer uses 3D visualization tools of ROS,which shows result in many ways.This system can be classified as single depth camera system or multi depth cameras system according to camera numbers.The former includes face detection and recognition,pedestrian detection and tracking;The latter overcomes the narrow FOV(field-of-view)of the single depth camera system by composing a camera network to relize long-term and cross-regional pedestrian tracking.This system uses fast face detection and recognition algorithms,which are easy to be deployed in low power consumption devices.For RGB cameras,Dlib face detector is integrated.For depth cameras,this thesis proposes a method of face detection based on Dlib trainer and RGB-D information,which detects faces(heads)accurately.Face recognition module uses eigenface and fisherface methods.For pedestrian perception,we use a traditional RGB-D detection algorithm and multimodal convolutional neural networks model.The former is powered by 3D image processing library PCL and the latter is based on faster R-CNN framework.The tracking module applies the idea of Tracking-by-Detection and takes advantage of the extended Kalman filter to achieve anti-occlusion results.At last,the system combines multi depth cameras to form a cam-era network.By means of camera calibration,the positions of all the cameras and the ground are known by each camera in the network.Thus 3D world could be reconstruct-ed by the camera network,which can be used to realize the long-term and cross-region pedestrian tracking.