Data-driven 3D Scene Viewpoint Mining And Assessment

As digital camera and smartphone become more and more popular, people are used to taking photos in their lives. Architecture is one of the most common scenes be-ing captured. But with lack of knowledge about photography and aesthetics, ordinary people usually have no idea about how to get an aesthetic architecture photo. In archi-tecture scenes, selecting a proper position and angle for capturing is one of the keys to get high quality photos. In this work, we will do research on the scene with one archi-tecture, which can help non-professionals with photo viewpoint selection. Specifically, out work mainly includes the following three aspects:(1) The algorithm to estimate viewpoint of architecture photos. Given a 3D mesh model with a number of photos about the scene, this algorithm can recovery camera matrix of every photo (on the coordinate of 3D mesh model). Firstly, a process called SfM (Structure from Motion) is conducted. It establishes the rela-tionship between photos by SIFT (Scale-invariant feature transform) descriptor matching. A partial point cloud model of the architecture is generated, with cam-era matrices of all photos(on the coordinate of point cloud model). Secondly, we sign a few of corresponding points on both models. Constraints are established base on these points, and transformation between these two models can be solved using Levenberg-Marquardt algorithm. Finally, by applying the transformation to the camera matrices produced by SfM process, we can get camera matrices of all photos on the coordinate of 3D mesh model.(2) Viewpoint clustering analysis of architecture scene. Camera position and camera pose can be derived from camera matrix. We make an interactive program to show these cameras within the architecture model coordinate.Which can offer non-professionals a reference for photo viewpoint selection. We apply Mean-Shift clustering algorithm to these viewpoints as a further study. This application efficiently shows the cluster of viewpoints.(3) A method of evaluating architecture photos. Machine-Learning is used as the main method to solve the task of architecture photo assessment. In the stage of feature extraction, besides 2D image features, the architecture model is rendered with photos’camera matrices and the model’s 3D mesh features of correspond-ing viewpoint are extracted. In the stage of classifier training, a Support Vector Machine is used for learning assessment data. Experiments are conducted and it shows that the learning method combining 2D and 3D features can lead to im-provement on the assessment task. This application can also help 3D designers to select a proper viewpoint for their scene rendering work.