Product Recognition Of Self-service Shelves Based On Deep Learning

In the new retail scene,there are a large variety of products of self-service shelves,which are susceptible to external factors such as light.In addition,customers' hands or bodies will block the key information of products when they hold the products.Therefore,only using image recognition algorithm in the natural scene cannot meet the application requirements of the self-service shelves.Aiming at the characteristics of the actual application scene of the self-service shelves,this paper identifies the handheld products using the human joint point positioning algorithm and image classification algorithm under the framework of deep learning and convolutional neural network.Firstly,human joint point positioning algorithm is used to accurately locate the joint points of the upper body of the customers.Secondly,the image classification algorithm mainly identifies the images which are centered on the joint points of the arm and contain the main features of the products.In order to realize the practicability of the algorithms,this paper studies the human joint point positioning algorithm and image classification algorithm from two aspects of speed and precision.The main research contents are as follows:(1)Research on lightweight human joint positioning algorithmTo identify products held by customers in the scene of self-service shelves,first of all,it is necessary to accurately locate the joint points of the customer's arms.In this paper,a Lightweight Convolutional Pose Machine(L-CPM)algorithm is proposed to improve the speed of human joint point extraction by introducing the lightweight convolution structure and reducing the resolution of the feature maps in the algorithm.In order to reduce the calculation amount of the algorithm,on the one hand,the lightweight convolution structure is used to replace the standard convolution structure of the human joint point positioning algorithm.On the other hand,the resolution of the feature map outputting by the partial convolution structure of the algorithm is minimized as much as possible by down sampling to reduce the redundant information under the precondition of retaining the main features of the image.The algorithm is tested by using the public datasets and the real scene datasets respectively.And the effectiveness of the algorithm is verified by evaluation indexes such as floating-point operations(FLOPs),visual effect and percentage of correct key-points of head(PCKh).The results show that the L-CPM algorithm can effectively reduce the complexity of the algorithm while ensuring the accuracy.(2)Research on human joint point positioning algorithm based on super-resolution reconstructionLightweight Convolutional Pose Machine(L-CPM)can reduce the amount of calculation.However,it inevitably results in loss of convolutional features and reduces the accuracy of human joint point positioning.Therefore,this paper proposes a human joint point positioning based on super-resolution reconstruction(EP-L-CPM),which can recover the lost feature information by introducing efficient sub-pixel convolutional neural(ESPCN)into L-CPM.ESPCN mainly acts on the last convolutional layer of the network,which rearranges the pixels of the low-resolution feature maps through the core operation of sub-pixel convolution to reconstruct the high-resolution feature maps.The algorithm is tested by using the public datasets and the real scene datasets respectively and the effectiveness of the algorithm is verified by evaluation indexes such as visual effect,percentage of correct key-points of head(PCKh)and floating-point operations(FLOPs).The results show that the EP-L-CPM algorithm can effectively improve the accuracy of joint point positioning without increasing the amount of calculation.(3)Research on image classification algorithm of self-service shelves based on attention mechanismThe recognition effect of products of self-service shelves mainly depends on the effectiveness of the training datasets and the performance of the classification algorithm.In terms of training datasets,the effective joint point is used as the center to intercept the images containing the main information of the products to build the training datasets with pure background and less redundancy.According to EP-L-CPM algorithm's ability to accurately locate the joint points of the hand holding the products,the position of the effective joint points is determined by using the joint points of the elbow,wrist and finger.At the same time,in order to increase the robustness of the algorithm and reduce the influence of external factors such as light,the training data set is further enhanced.In terms of algorithm performance improvement,this paper proposes an image classification algorithm ATN-Mobile Net based on the attention mechanism,and it mainly introduces an attention model at the back end of the Mobile Net V2 to enhance the characteristics of the attention area.In order to verify the effectiveness of the proposed algorithm,multiple sets of comparative experiments are performed on the constructed data set.The results show that the proposed algorithm can effectively improve the accuracy of product recognition.