Craniofacial Feature Point Calibration And Registration Based On Deep Learning

Three dimensional data registration is the basic problem of computer vision.The so-called registration is to transform the target data to make the accurate registration between the target data and the reference data.It is the basis of craniofacial reconstruction,gender identification,ethnic classification and so on.Different shape and posture of 3D craniofacial data,different acquisition methods and environments lead to inconsistent points,and some data have non rigid deformation.It is difficult to define and extract features of craniofacial data,so it is a challenging work to obtain accurate matching results.Due to the continuous development and wide application of convolutional neural network in recent years,we apply it to feature point detection.In order to solve the problem that it is difficult to input 3D skull and facial data directly into the existing 2D deep learning network,this paper proposes a model representation method of using depth map to process 3D facial data and curvature map and information map to process 3D skull data,which simplifies the model representation and uses automatic calibration network to predict feature points,It reduces the tedious work of manually labeling feature points.Finally,the predicted feature points are used as the target data control points for Thin Plate Splines(TPS)registration to achieve automatic and accurate craniofacial registration.The main innovation and research work are as follows:1.Automatic calibration of feature points based on depth mapIn this chapter,an automatic calibration method based on depth map is proposed for3 D face data.The depth map is used to express complex 3D face data.According to the physiological characteristics of 3D face,seven points are selected as feature points,including left and right outer corner points,left and right inner corner points,nose tip points and left and right mouth corner points,An automatic calibration network of feature points is constructed to estimate the coordinates of feature points.Finally,the predicted coordinates of two-dimensional depth image are converted back to three-dimensional data to obtain the accurate position of feature points.2.Automatic calibration of feature points of 3D skull data based on curvature map and information mapCompared with 3D face data,3D skull data has more complex topological structure.According to the physiological characteristics of 3D skull data,eight feature points are selected,including left and right outer orbital points,left and right inner orbital points,left and right nasal orbital points and left and right mouth corner points.Due to the complexity of the surface curvature of skull data,three-dimensional skull data representation methods based on curvature map and information map are proposed.The three-dimensional skull data is transformed into two-dimensional image data with rich three-dimensional feature information and input into depth learning network.At the same time,the three-dimensional feature points marked according to physiological features are transformed into two-dimensional coordinates,and finally the feature points of three-dimensional skull data are predicted by network and applied to skull registration.3.3D craniofacial registration based on deep learning prediction feature pointsThe feature points predicted by the automatic location method of 3D feature points based on depth map are used as the control points of 3D face target data,and the feature points predicted by the automatic location method of 3D skull data feature points based on information graph are used as the control points of 3D skull target data.The feature points of reference data and the feature points predicted by target data are used as the control points of TPS registration.The mapping of reference data to target point set is obtained.TPS transformation is carried out for reference data.Then,the nearest point of target data from reference data is found on this basis,and finally the registration result is obtained.By comparing with other methods,we can see that our method has higher registration accuracy.