Research On Recognition Of Supermarket Commodity Based On RGB-D Multidimensional Information Deep Learning

Using intelligent equipment such as service robots to improve service quality and efficiency,thus realizing automatic management is the direction for future development of supermarket.The commodity identification technology is an important part of the supermarket intelligence process.The deep convolution neural network(DCNN)of deep learning can achieve good results in the field of image recognition,which mainly depends on the advanced nature of the DCNN algorithm and sufficient training data.However,the model trained using the images of the commodity collected in the ideal environment do not work well in practical applications.Therefore,this paper collects commodity images in a real environment.Due to the limited range of angles collected in the real environment,the dataset does not contain comprehensive information and the number of training samples is limited.In addition,the background of the sample image is complex,and there may be problems such as occlusion and extrusion deformation between the products.Therefore,how to use the images collected in the real environment to complete identification becomes a challenge on the research.To solve these problems above,the paper proposes a supermarket commodity recognition algorithm based on RGB-D multi-dimensional information deep learning to achieve fast and accurate identification.The main research of this paper is as follows:(1)Aiming at the insufficient training samples collected,data augmentation is applied by image rotation and the transfer learning of DCNN is applied on the dataset to improve training efficiency and recognition accuracy.The specific process of transfer learning is to use a network parameter model that has been trained on ImageNet,a large-scale image classification dataset and then transfer the parameter model to the small-scale dataset of supermarket commodities through network fine-tuning.(2)In order to improve the DCNN algorithm,aiming at the problems of supermarket commodity images with dense pixels and rich details that lead to the loss of extracted features,an image recognition algorithm is proposed based on Multi-scale Convolution Kernel Convolution Neural Network(MCCNN).MCCNN uses a new multi-scale convolutional kernel model to strengthen feature extraction.This model contains convolution kernels of different sizes and uses cascaded 3*3 convolution kernels instead of larger convolution kernels.This design can retain the details of the feature extraction process and reduce network parameters.Meanwhile,the model uses the 1*1 convolution kernels to realize nonlinear transformation in different dimensions,which can increase the feature expression and extraction capability of the network.(3)In order to get more available information of the training data,aiming at the limited information of the training samples,a supermarket commodity recognition algorithm based on RGB-D multidimensional information deep learning is proposed by adding the depth information of commodities.The multidimensional information includes both color information and depth information of the commodities,and also includes different fusion information obtained when the two kinds of information are integrated at different levels in the DCNN.In the process of information fusion,the involved network splices results in overmuch network parameters.To fix the problem,a GAP+FC(Global Average Pooling + Fully Connected,GAP+FC)network structure is designed to optimize the structure and effectively reduce the number of network parameters.(4)An RGB-D supermarket commodity dataset with depth information is established and calibrated.A network based on RGB-D multidimensional information deep learning is built,and a network parameter model that can be used for automatic identification of supermarket commodity is trained by the network.Finally,the design and implementation of automatic identification system for supermarket service robot is completed.