Single Image Based Facial Appearance Acquisition System

We present a system to acquire facial skin material based on single image,that can extract facial reflectance image and subsurface scattering parameters,which can be used in real time rendering after facial 3D reconstruction.Our system consists of three major parts:highlight removal,intrinsic image decomposition and subsurface scattering regression.For the input facial image,we start with 3D reconstruction with bilinear face model,to acquire face marks and geometry model.Since there might be undesired global illumination,we then process highlight removal for the input image,and restore lost skin color in overexposed area.Then,with the prior of facial region and skin color,we process intrinsic decomposition using optimization,acquiring reflectance image and shading image.To render face model more efficiently,we train a convolution neural network to regress subsurface scattering parameter.Finally,we render a real-time face model with reflectance texture and subsurface scattering parameter using subsurface scattering rendering model.In summary,our highlight removal and intrinsic image decomposition algorithms combine existing methods and facial priors,resulting better results for human face,and we introduce subsurface scattering rendering method and regress subsurface scattering parameter with convolution neural network for more reality rendering result.