Research And Implementation Of Visual Capture And Tracking Technology In Unmanned Store Scenes

With the continuous change of consumption trend and competition pattern,convenience stores' terminal mode innovation,commodity combination,business extension way,digitalization degree,innovation of supply chain management and the expansion of convenience stores all need to be innovated.Convenience stores are gradually developing to the direction of intelligence and network.At present,the general store will have a salesperson or cashier,through the salesperson or cashier to settle the account of the user's purchase of goods.However,when a large number of users buy goods,they often need to wait in line,so that it takes a long time to pay the bill.In addition,labor costs of salesmen or cashiers need to be provided.Existing unmanned store system needs to set up access control at the entrance or exit.When users enter or leave the store,they need to wait for a period of time for face recognition.Only when face recognition is successful,customers can enter the store for shopping.Setting up access control at the door increases the hardware device,requires more money,and consumes customers' time.In this paper,we design and implement a visual capture and tracking system architecture based on the demand analysis of the real unmanned store system.The architecture can be divided into four parts: fetch-put-back judgment,commodity type identification,establishing the relationship between goods and people,and identifying people's identity.It can run on the general x86-64 architecture computer.In the fetch-put-back judgment module,this paper trains the action discrimination model and the target detection model.By detecting the video frame by frame,the action discrimination model can find the key frames in the video.Target detection model can detect whether there is a commodity in the hands of the customer before and after the key frame,so as to judge whether to take or put back the action.In the function module of commodity type recognition,two models are trained,one is target detection model,because the proportion of commodity in the graph is very small,in order to identify accurately,it is necessary to cut out the commodity from the graph,and the image of commodity can be obtained by using the target detection model.The other model is the commodity classification model to determine the specific category of goods and get the ID of goods.In the function module of establishing the relationship between goods and people,the pedestrian re-recognition model is trained after the person operating the goods is determined,and the pictures of the person under the camera from different angles at the same moment are obtained.In the identification function module,the face recognition algorithm is designed and implemented to get the customer's ID.A shopping list is obtained based on the item ID and customer ID of each action.This paper tests the visual capture and tracking system in the scene of unmanned store from two aspects of function and performance.Functional test results show that the system has achieved all the functions in the requirements.The performance test shows that the system achieves the expected F1 score.We analyzed the test results and gave the content that can be improved.