Research On Keypoint Based Method For Supermarket Items Detection And Recognition

With the development of self-service retail,the detection and recognition of supermarket items have attracted much attention of domestic and foreign researchers.Deep learning is an effective way to improve the accuracy of items detection and recognition,of which keypoints based detection and recognition algorithms are of great value to research and application.First,an object detection algorithm framework based on keypoints is proposed.To avoid the problem of inaccurate object positioning caused by the classification confidence criterion,the joint representation of positioning and classification confidence is used in the algorithm framework.The high-level feature prediction may ignore the other levels of information,an adaptive multi-level feature fusion module is thus proposed to maximize the effective information by fusing features from different levels of the network.In addition,to address the problem of insufficient feature extraction capabilities of the network,a global context module is introduced to aggregate global context information of each object to expand the receptive field of network model and to improve its capability of feature extraction.Experimental results on the MS-COCO dataset show that the proposed method effectively improves the accuracy of object detection.Secondly,an algorithm framework based on multi-scale weighting and spatial attention module is proposed to realize shelf items detection.In view of the size difference of the items in the shelf image,the multi-scale feature extraction and weighted fusion module is embedded into the network,and the convolution kernels of different scales are used to extract multi-scale features,and realize the weighted fusion.Generally,the shelf items in the image are small and a spatial attention module is proposed,which uses the low-level features to generate an attention map and to enhance foreground information of objects and their positioning accuracy.Meanwhile,the items on the shelves are placed closely,and a method to compute the item angle is proposed to avoid the overlap of the bounding boxes of items.Experimental results on the SKU-110 K dataset show that this algorithm framework improves the accuracy of shelf items detection.Finally,a detection algorithm framework is proposed based on cross-stage local connection,spatial pyramid pooling and counting branches.By introducing cross-stage local connection,the redundant gradient of the network is decreased while the network learning ability is improved.Moreover,the memory occupation and resource consumption can be reduced.In addition,to overcome the problem of the missing detection caused by insufficient network receptive fields,the spatial pyramid pooling is introduced to expand the receptive fields and does not introduce more parameters.Furthermore,to accommodate to the item shapes and improve the detection reliability,a deformable convolution is used to construct the counting branch,which corrects the detection results.Experimental results on the RPC dataset show that the proposed framework is able to improve the detection and classification accuracy of items on the checkout counter belt.The thesis mainly focuses on the problem of detection and recognition of supermarket items.An algorithm framework based on keypoints is introduced and the ablation experiments have been done on several datasets.The results validates the detection performance of small different items placed closely.The proposed algorithm frameworks are able to be used in new retail such as automated retail stores and cashier-free supermarkets.