A Research Of Real Sense 3D Face Reconstruction And Measure System Based On Multi-view Stereo Vision

With the social development, there is growing emphasis on beauty, maxillofacial plastic became popular. However, dentist desires an effective solution to make preoperative assessment for the patient who needs maxillofacial plastic. It is difficult to show the patient a visual therapeutic processes or make accurate surgery predictions. Dentists urgently need a facial three-dimensional imaging and measurement systems with high-accuracy and easy to operate, for virtual plastic surgery.This paper systematic reviewed artworks in computer vision field, proposed a real sense 3D face reconstruction and measuring system based on multi-view stereo vision. The system is a passive stereo vision reconstruction technology. It is non-invasive, non-interference with no mark, no face paint, and no structured light. Meanwhile, the system can capture people’s face image instantaneously, single shot capture can reconstruct a real sense and high-quality face model. Subjects almost without waiting for the acquisition to be completed. The main work and innovation are as follows:(1) In this paper, we designed and produced a multi-view stereo vision hardware devices. The hardware devices armed with five consumer digital cameras and four soft lighting box,and a mirror was set on the top of the camera which facing the subjects, hold the camera optical axis perpendicular to the surface of the mirror. The system is tested with a number of subjects, have achieved good results, and we compared the results with laser scanning, the average error is less than 0.370 mm. This shows that our system is reliable.(2) This paper proposed a front view posture multi-view camera calibration method. We draw lessons from the doctor examine a patient, placed a mirror on the top of the camera which facing the subjects, hold the camera optical axis perpendicular to the surface of the mirror. Testers were required to gaze his own eyes in the mirror when capturing images. After calibration, we can get a front view face posture model. The front view posture is helpful in medical applications such as surgical simulation.(3) This paper proposed an improved stereo matching algorithm based on hierarchical pyramid. Improvement of this paper is to set the cameras to collect data symmetrically. Use opposite data to fill holes that result from occlusion when matching one side images. The algorithm iteratively produce higher resolution matching from low resolution matching. After each iteration of the process, the disparity map becomes more refined.(4) This paper review the fundamental shading method in computer graphics fields, implement a shading constrained energy minimization mesh de-noising algorithm, applied image RGB information in mesh de-noising. The method can recovers details of mesh, overcomes the traditional methods’ shortcoming of excess smoothing.(5) This paper proposed an interactive method extracting facial contours. As user draw a plane on the model, than the method result a contour line of the face model. The results showed that the method is simple and reliable. The method may be applied in face feature analysis, simulation applications such as orthodontic surgery.(6) We developed a software system on the windows platform based on OpenCV, OpenGL, Vcglib, etc. computer vision C/C++ libraries. Our software implements multiple practical utility: face image capture, storage, reconstruction and measurement. The workflow of our system includes: stereo vision calibration, image capturing, stereo matching, data merging, surface reconstruction and texture mapping. After calibration, the system is completely automatic and does not require adjustment software parameters for individual people or manual processing, the system automatically reconstruct a face model. Whit the measuring software, we achieve a lot of features, include: length of two points, geodesic distance of two points, counter line curvature and etc.