Research On Segmentation And Recognition Of Products On Grocery Shelves

Common supermarket management systems mostly aim at checking inventory of products.But operations on shelf products are mainly manual,leading to poor efficiency and high cost.Thus intelligent analysis on shelf images will bring great benefits to retail industry.To solve this problem,a novel method of segmentation and recognition based on SIFT features is proposed in this paper.The types and amount of products on shelves can be analyzed using this method.Thus it will improve the efficiency of shelf management.This paper consists of two parts: segmentation and recognition.Images of single product are extracted and recognized effectively.The main work of this paper is as follows.(1)Segmentation of shelf images includes two steps: segmentation of single layer images and extraction of single product images.The interlayers in shelf images are usually in light and uniform colors with clear edges.Therefore,we firstly use Hough transform to detect the rough position of segmentation lines.Then projection of gray images and edge images is used to verify the previous results,and shelf images are segmented into single layer images by affine transformation.We proposed a novel method for extraction of single product image based on SIFT features.After evaluating the similarity of local features,key-points are clustered into visual words by forward selection method.Then every visual word is assigned into several duplicate objects.Each allocation scheme is estimated according to structural similarity.The best scheme is selected through continuous iterations,and single product images are extracted based on it.(2)Products with same category usually have high similarity and are difficult to distinct.We extract local features from single product images and compare them with all the samples in dataset.The sample image with the most matching points is considered as the recognition result.Rough matching key-points are selected according to the ratio of distances.We also proposed a new method based on geometric constraint to eliminate the mismatching points.Firstly,the angle differences of the key-points between sample image and single product image are calculated in order to remove some mismatching points.Then the ratios of the pairwise distances between reference image and observation image are used to mark the abnormal ratios,and the mismatching points are removed by majority vote method.Our experiments are based on the previous segmentation.Experiments are carried out on our dataset and public dataset GroceryProducts to prove the feasibility.The results show that our method has increased accuracy by 4.4% and 10% and thus verifies the efficiency of the proposed method.