Joint RF-Image Object Detection Algorithm

In recent years,there is an increasing demand for Location Based Services(LBS).In order to adapt to different LBS scenarios,researchers have proposed multiple solutions.Specifically,RFID-based location technology is one of the solutions,and has been widely used in retail stores,logistics warehouses and other scenarios.Generally,the RF signals used by RFID carry a small amount of information,which are only suitable for coarsegrained location services.On the other hand,with the development of deep learning,image-based object detection has also been widely studied.Compared with RFID-based algorithm,the image-based algorithm is less robust due to the various effects such as lighting condition and occlusion.In order to make full use of the robustness of RF signal and the fine-grained characteristic of image signal,this thesis focuses on the joint RFimage object detection algorithm.Firstly,the RFID-based positioning algorithm and the image-based object detection algorithm are implemented.Then,the multi-modal algorithm is designed to fuse RFID information and image information.As a result,our multi-modal object detection algorithm can not only improve the detection performance,but also provide a new idea for automatic image annotation.Specifically,RFID tag can uniquely identify the object that needs to be labeled.Then,one can detect and label specified object in specified scenarios with RFID reader and camera.The main content of this thesis can be divided into three parts: 1)In the RF part,we analyze various RF location technologies in detail,and choose RFID to implement our AOA estimation algorithm.Firstly,we implement the RFID reader client based on the low-level reader protocol,and successfully obtain the key datas such as RSSI,RF phase,and so on.Then,we model the datas collected by virtual antenna array as AOA estimation problem,which can be solved by the MUSIC algorithm.Finally,we verify the feasibility and effectiveness of the ideas mentioned above through extensive experiments.2)In the image part,we analyze various object detection algorithms in detail,and choose the Onestage Grounding algorithm as the basis of our object detection task.In order to fuse object category information,we use NLP model to extract the category featrue.In order to improve the detection performance,we use attention mechanism to filter visual feature map.At the same time,according to the storage characteristics of RFID tag,we propose an attribute fusion algorithm,which can identify different objects based on the visual attributes such as shape,material and color.Finally,we verifiy the feasibility and effectiveness of the algorithms mentioned above through experiments.3)In the joint algorithm part,we utilize the idea of transfer learning.Specifically,the AOA spatial spectrum of the existing dataset's image instance is simulated.Then,we use attention mechanism to fuse AOA spatial spectrum and image feature map.Finally,we implement a multi-modal object detection algorithm for object category information,AOA spatial spectrum information,quantity information,and image information.Through the works mentioned above,this thesis implements a joint RF-image object detection algorithm.The method proposed is feasible and effective in real experiments,and provides new ideas for further multi-modal object detection tasks.