Research On Viewpoint Recommendation For Photographing Architectures And Facial Attractiveness Enhancement In Photo With Multi-view Constraint

Computational photography is a research field that integrates computer vision,graphics etc.It has received considerable attention due to the increasing popularity of digital photography and has been widely used in image blending,image enhancement,and photo style transfer etc.With the popularity of digital cameras and smart phones,users pay more attention to taking the photos with high quality.Lots of the available photos on the Internet are involving the architectures and portrait,especially when visiting the famous tourist attractions.This paper focuses on the research of viewpoint assessment and recommendation for architectures and facial attractiveness enhancement,and aims to provide users with the guidance on viewpoint selection of architectures and automatic facial attractiveness enhancement.To this end,the correlations of 2D image features and 3D geometry features with respect to the viewpoint of architecture are explored with multi-view constraint for viewpoint assessment and recommendation.Besides,the multi-view constraints between geometry structures of the geometry adjustment result and the appearance adjustment result are explored for the facial attractiveness enhancement.This paper provides advices for viewpoint selection of architectures before taking photos,and assesses the viewpoint after photographing.At last,the attractiveness of portrait in the photo can be automatically enhanced in a data-driven manner.Specifically,this paper includes the following aspects.1.Viewpoint registration and preference analysis of architecture photos in a datadriven manner.Given photos of the same architecture,the relative position between the photos are first explored with Sf M algorithm.At the same time,Sf M generates the coarse point cloud model with these photos.The model registration and transformation transfer between the point cloud model and the mesh model are conducted to get the viewpoint of each photo with respect to the mesh model.Compared to the method of single photo registration,this manner greatly reduces the burden of viewpoint calibration and achieves high accuracy.By this way,photos of 15 world famous architectures are collected and calibrated.For the camera external parameter of the viewpoint,viewpoint clustering is conducted with the distance defined on the special European group.The clustering efficiently distinguishes the photos in viewpoint verifying the viewpoint preference when photographing,and provides a way to quickly discover common viewpoints for a famous architecture.2.Viewpoint assessment and recommendation for architectures with multi-view learning.The goodness of the viewpoint for an architecture is related to the 2D image features in the photo and the 3D geometry features revealed in the viewpoint.A series of 2D image features involving viewpoint information as well as image quality and a series of 3D geometric features are proposed for viewpoint assessment and recommendation.To explore the relationship between the 2D image features and the 3D geometry features,multi-view learning is implemented to learn the viewpoint goodness with features in 2D image and 3D geometry aspects.The method first verifies the effectiveness of individual features as well as the combination of the two view features,and it also suggests that multi-view learning achieves the better performance.Given an architecture photo,we can utilize the 2D image features revealed in the photo to achieve viewpoint assessment and recommendation.For the photos of an architecture,we can reconstruct its 3D model to make viewpoint recommendation.Given 3D model,we can sample the viewpoints and utilize the 3D geometry features and the 2D image features in the rendered image to achieve viewpoint assessment and recommendation.For the world famous architectures,users can provide only few photos to get recommended viewpoints.3.Facial attractiveness enhancement with multi-view constraint.Facial attractiveness enhancement involves the facial geometry structure adjustment and the facial appearance adjustment with generative adversarial netwroks,which we call“FA-GANs”.Our facial attractiveness enhancement method contains two branches for these two aspects,and makes the enhancement by exploring the feature space of attractiveness faces.Specifically,each branch is with the generative adversarial networks to make the enhancement,respectively.A facial attractiveness ranking module is embedded in the appearance branch,and the discriminator can judge the generated facial attractiveness in a coarse-to-fine grained way.For the face enhanced by the appearance branch,the geometry structure should be consistent with the adjustment result of the geometry branch.To this end,the multi-view constraint is proposed to guide the enhancement in both aspects.The experiments show that FA-GANs can effectively make the face attractiveness enhancement and achieve better performance than the state-of-the-arts.