Fine-grained Object Classification Based On Pose Normalization

Fine-grained object classification,as an important part in traffic monitoring system,plays a vital role in real-time vehicle type identification.For vehicle targets,fine-grained classification needs to classify hundreds of subcategories in the category of vehicle to identify vehicles of different types.In ideal situation,it can be achieved by detecting license plate and checking from system.However,due to the diversity of environment and license plate,this algorithm does not guarantee fine-grained classification accuracy.Currently,fine-grained classification algorithms are divided into two categories.One is for general objects and the other is for vehicles.On one hand,general fine-grained classification is based on parts of objects which may be discriminatory and provide lots of information for fine-grained classification task.However,this kind of method doesn't introduce optimizing constraints for vehicles.On the other hand,fine-grained classification methods for vehicles use 3D structure information to improve fine-grained classification results.There are two existing problems in such a kind of method:high cost of 3D model annotation and difficulty of vehicle pose estimation.In this paper,we propose a fine-grained object classification algorithm based on pose normalization.Fine-grained classification task is divided into two related sub-tasks: vehicle's pose estimation task and classification task based on pose normalization.3D bounding box is firstly estimated,which helps to normalize vehicle's pose.The fine-grained classification is conducted on the pose-normalized image.Normalized vehicle image can effectively reduce the within-class difference caused by object's pose,which helps to improve the accuracy of fine-grained classification.As a result,two innovations are proposed in this paper: the generation method of vehicle's3D bounding box and the pose normalization method.The first improvement is to build a Bi-UNet feature fusion network for3 D bounding box's heatmap generation in an end-to-end manner.This network is improved from a block map generation network based on U-Net network.The keypoints' coordinates are extracted from generated heatmap.This method does not need 2D bounding box and vehicle contour information as the input of network.A skip connection structure between different subnetworks is added in Bi-UNet network,which can effectively integrate features between different subnetworks.Experiments show that our network outperforms other state-of-art 3D bounding box generation methods.The second improvement is to build a pose normalization method.Three surfaces are extracted from generated 3D bounding box,and each surface is normalized into a 2D plane by perspective transformation.It effectively reduces the within-class error caused by pose variations.We also propose an improved pose normalization method according to actual structure of vehicle.An additional bevel surface is transformed and added to the normalized result.With normalized input images,different classification networks are used to get fine-grained classification results.Our fine-grained classification algorithm directly completes pose estimation through the vehicle image without any prior pose information.On Box Cars21 k dataset and Box Cars116 k dataset,our network outperforms the state-of-arts with 9.35% and 7.15% improvement of Pck(Percent of correct keypoints).As a result,the accuracy of fine-grained classification is improved by 1.81% and 1.45%.