| With the continuous development of artificial intelligence technology,person search plays an essential role in many application scenarios,such as video surveillance,security and intelligent transportation,and has therefore received extensive attention from academia and industry.Text-based person search aims to retrieve target persons in a large-scale person image database using text as the query information.It is crucial to explore fine-grained semantic alignment between person images and text for this task.This thesis focuses on the research of text-based person search based on fine-grained semantic alignment,and the main research contents are as follows:(1)Existing methods only consider global matching or use computationally expensive local matching methods based on cross-attention.To address this issue,this thesis proposes a text-based person search method based on semantic alignment feature learning,which adaptively learns semantic alignment image-text local features.The method can aggregate image block features in the image into image region features containing specific semantics,and aggregate word features in the text into noun phrase features,and adaptively achieve semantic alignment between image region features and noun phrase features in the text.Experimental results show that the proposed method outperforms other methods on text-based person search tasks and has good generalization ability.(2)Existing methods lack prior knowledge as supervised information and are,therefore,difficult to deal with issues such as occlusion,background interference,and diverse poses.Therefore,this thesis proposes a text-based person search method that incorporates pose information as prior knowledge,extracts keypoint features of the human body through pose estimation methods,and integrates the keypoint features of the human body into semantic alignment feature learning to optimize the learning of image-text features.Experimental results show that the proposed method performs significantly better on various text-based person search datasets.(3)Existing methods often use single-modality encoders to encode images and text separately,lacking the ability to handle multi-modal data.Therefore,this thesis transfers the general image-text knowledge of cross-modal pre-training models to fine-grained text-based person search tasks to improve feature expression ability.In order to bridge the gap between coarse-grained models and fine-grained tasks,this thesis proposes a visual grounding-driven pre-training text-based person search method.For noun phrases in the text,the model effectively locates the corresponding image regions to enhance cross-modal semantic alignment.Experimental results show that through visual grounding,cross-modal pre-training models can effectively improve performance on text-based person search tasks.In summary,this thesis proposes a series of solutions to achieve fine-grained semantic alignment in text-based person search tasks.Experimental results show that the proposed method has significant performance advantages over existing methods. |
