Upright Orientation Of 3D Shapes With Convolutional Networks

Most objects are usually posed in their upright orientations, which makes them easily recognizable. Also, it is the very first step to pose the given 3D shapes in their upright orientations in many graphics tasks, such as matching, retrieval and shape anal-ysis. Moreover, it can be used to generate recognizable object thumbnails, helping the management of 3D shape repositories. Due to various reasons, many models in existing databases are not in their upright orientation.In this paper, we present a learning based method to predict the upright orienta-tion using 3D Convolutional Networks (ConvNets). Given voxel representations of 3D shapes and corresponding orientation vectors, this prediction task can be formulated as a regression problem. Leveraging the learning ability of deep neural networks, general categories of 3D shapes can be handled without making any assumptions such as sym-metry or parallelism. Besides mesh models, the proposed method can deal with shapes represented in other types that can be voxelized, such as implicit surfaces and point clouds.Compared with the ConvNets based approach, existing methods are limited by their predefined rules. Nevertheless, this observation is not applicable to all shapes. Thus learning based methods are appreciated to deal with general objects. Although the idea of data-driven has been adopted in existing approaches, the learning procedure is based on the hand-crafted features such as stability, visibility and parallelism, which are not suitable for general 3D shapes. By contrast, neural networks work in the style of end-to-end learning. High-level knowledges can be captured from raw data, without relying on object’s regularity such as explicit symmetry.However, a single ConvNet does not work well for all types of shapes. The key challenge is that each shape category exhibits particular characteristic on the upright orientation. This is referred to as interference effect which will lead to poor generaliza-tion. In other words, different strategies should be taken to handle diverse categories. Thus a divide-and-conquer scheme is used in our system. Each shape is first classified by a network and then fed into one of the orientation regression networks that are trained on each of the categories. Furthermore, a distance based clustering method is proposed to reduce the number of networks and a novel test-time augmentation procedure is used to improve the accuracy.The efficiency and effectiveness of this approach are demonstrated by extensive experiments. Our system achieved the accuracy of more than 90% on the test data and showed the generalization capability of inferring upright orientations for shapes not belonging to the training categories. Also experimental results showed that our system is able to handle several cases that other methods fail. Moreover, estimation for each shape took no more than 0.15 s on average, which is much faster than existing approaches, thus applicable to robotics tasks in which immediate feedback is required.