Processing Method Of Single-frequency Fringe Patterns Phase Map Using Deep Learning

Active three-dimensional 3D vision methods have the characteristics of fast measurement speed,non-contact and high accuracy and are widely employed for 3D reconstruction in industrial design,industrial detection,and robots.Compared with other active 3D vision methods,fringe-projection-based active 3D vision methods can achieve higher accuracy by projecting single-frequency or multi-frequency fringe patterns.Phase unwrapping is the essential part of the three-dimensional vision with fringe projection.Phase unwrapping is recovering the continuous phase value from the wrapped phase value of [-?,?).Phase unwrapping methods are classified as spatial phase unwrapping(SPU)and temporal phase unwrapping(TPU).SPU methods require only single-frequency fringe images,which is helpful to the real-time 3D reconstruction.However,SPU methods suffer from fundamental drawbacks,including accumulation and propagation of phase errors along the unwrapping path,failure to unwrap phase maps with multiple isolated regions,and the output of relative phase value instead of the absolute phase.TPU methods overcome the above drawbacks of SPU by projecting multi-frequency fringe patterns at the cost of measurement speed.For the 3D vision system of fringe-projection using TPU methods,to realize the real-time 3D construction,the projector and camera are required to operate under the high-speed model.Therefore,improving the spatial phase unwrapping method and retrieving the absolute phase from single-frequency fringe patterns has vital significance for reducing equipment frame rate requirements,improving real-time performance,and achieving system miniaturization.This thesis proposes a deep learning-based single-frequency The phase map processing method,the main work is:(1)Firstly,this thesis proposes a learning-based approach to detecting unreliable points in the single-frequency phase map.Traditionally,different types of unreliable points are handled separately with different well-designed methods.Inspired by deep-learning-based images applications,this thesis proposes a "one-to-many" strategy that trains a neural network to analyze phase maps with different unreliable points.The detection of unreliable points in phase maps is formulated as a problem of three-class pixel classification.The three classes are background points,reliable object points,and unreliable object points.The ground truth of the three classes points is labeled automatically.After being trained properly,this neural network can detect a variety of unreliable points in the phase maps of complex scenes,including image defocus,motion blur,phase discontinuity,low surface reflectivity.(2)Secondly,this thesis proposes a method of recovering absolute phase from single-frequency fringe patterns using deep learning.Phase unwrapping is assigning a unique integer to each pixel,can be regarded as a pixel classification problem.On the basis of the difference between phase maps and the images which general image semantic segmentation addresses,the general semantic segmentation network is modified,and a new phase unwrapping network is proposed.The phase unwrapping network takes as input the single-frequency phase map without removal of unreliable points.The ground truth of training is obtained through TPU.Compared with existing learning-based phase unwrapping methods,the presented network achieves the absolute phase recovery from one single-frequency phase map and handles phase maps of 1024×1024 pixels at 30 fps on a GTX 1660 Ti.(3)Thirdly,this thesis proposes a shift-sampling method is proposed to handle the high-resolution phase map with deep neural networks.The DCNNs designed for general image-segmentation tasks often fail to load high-resolution images for training.Shift-sampling is proposed to convert a high-resolution phase map into a small phase map concatenated with multiple channels,and the small phase map still contains the global information of high-resolution phase maps.Based on the above-mentioned method,this paper realizes the 3D reconstruction of the real dental model on the prototype of the handheld 3D scanner of small objects,and realizes the prototype system of handheld fringe structured light3 D imaging based on deep learning,which verifies the feasibility of this method.(4)Finally,the above-mentioned methods were verified on a large-scale dataset with many challenging scenarios.Continuous 3D reconstruction of the real dental model was realized on handheld 3D scanner prototype,which indicates potential applications for 3D measurements of complex scenes with fringe-projection systems of single-camera and single-projector.