Personalized Hand3D Modeling Based On Multi-view Stereo And Iterative Deformation

The hand is a very important interface for humans to interact with the physical world. In virtual reality and gaming applications the use of a personalized rather than generic hand model can increase the sense of immersion and in some cases may lead to more effortless and accurate interaction with the virtual world. However, it is very challenging to construct a personalized hand model because of its complex shape and lack of strong features, especially with low-end devices and without markers.In this paper, we present a hierarchical approach for constructing a personalized hand model with web cameras and without markers. Our general idea is to deform a generic template step by step under the different granular features. Contour convexity defects (point level), contours (line level), as well as SIFT, SURF and Sobel response (surface level) are integrated to depict the user’s hand from distinct levels. Notice that in low-solution images both the significances and the reliabilities of these features reduce gradually, and that the accuracy of the correspondences for the next deformation depends on the result of the previous result. Thus as the deformation processing, the template becomes more and more similar to the user’s hand, whilst the features required getting less and less strict. In this way, we make a reasonable tradeoff between low-end equipments and high accuracy.The main contributions of this dissertation include the following three aspects:(1) A new framework is proposed to construct the personalized hand3D model. Due to the low-cost hardware and the lack of strong features in images of user’s hand, it is hardly to reconstruct the model via the standard stereo vision. We extract different features from images and build a3D point cloud by feature-based stereo vision. A generic hand template is deformed gradually under the guide of the point cloud, and turns to be the user’s hand model finally.(2) An iterative deformation method is applied by variational thresholds to handle the interdependent relationship of mutual coupling between the mesh deformation and the correspondence computing among the template and the point cloud.(3) An approximate approach is proposed to recovery the positions of contour points in the template from a single image. In order to maximize the use of pixels to get more details, the user’s hand should be as large as possible in all the images which leads to a short distance between the cameras and the user’s hand. And also because of the perspective projection, the hand contour in the left image is quite different from that in the right image. It is incorrect to obtain the positions of contour points by matching them simply. According to the projection principle, these positions are recovered by assuming that their depths remain unchanged after the deformation based on the point level features.