Research On Clothing Recommendation By Fusing User Social Information And Clothing Matched Knowledge

With the fast development of social fashion websites(such as Pinterest and Mogujie),online shopping is becoming a convinent and attractive way for users.However,facing a large scale of goods online,people often spend much time and energy on which one to buy,which badly disrupts the user experience.Recommendation System aims to quickly recommend goods which meet users' requirements.Different from recommendation on books or movies,the clothing item has its own characteristic.Visual content of clothing items is an essential factor to determine user decision making,which is often ignored by previous works on RS.In addition,User social information is another key factor for user decision making.Furthermore,clothing matched knowledge can improve the performace of recommendation and is great helpful to better clothing sales.We treat clothing data and the records of user shopping as the main research targets,and focus on clothing image semantic annotation,clothing matched knowledge discovery and personalize clothing recommendation.Our work are mainly listed as follows.First,a part-based clothing image annotation by visual neighbor retrieval is proposed in this paper.Previous works focus on image annotation in object level,such as image category.For clothing image,users often pay attentions on the description for details.Clothing image annotation is a challenging task due to large variations in clothing appearance,human body pose and background.Therefore,the similar image search is first conducted to discover visual neighbors for a query image.The impact of large variations of clothing is alleviated by pose detection and part-based feature alignment.Tag relevance and tag saliency are taken into consideration to obtain the candidate tags.The relevance of candidate tags is identified by mining visual neighbors of a query image,while the saliency is determined according to the relationship between query image parts and part clusters on the whole training set.Clothing image annotation is implemented according to the relevance and saliency of candidate tags.Next,a multi-label regression convolutional neural network is proposed for clothing attribute prediction.Clothing attributes are middle-level representations for clothing items.Attribute prediction is a challenging task due to large variations in clothing appearance,models' pose and background.Furthermore,the region of clothing attribute in clothing images is usually small,which make feature learning more difficult.Though deep features are better than traditional features on representations for general images,they are still not enough for clothingimages,since they ignore the relationship between clothing attributes.Therefore,this kind of relationship is mined with a multi-label regression model and is integrated into a multi-branch convolutional neural network.In addition,a novel multimodal neural network is proposed to learn compatibility between clothing items.fashion compatibility is beneficial for clothing recommendation.Recently,learning fashion compatibility increasingly attracts more and more attentions and has become a very hot research topic.Existing works explore this problem only depending on visual features.However,semantic gap is still a unsolved issue,while fashion compatibility modeling belongs to a semantic level task.Therefore,we simultaneously integrates both semantic and visual embeddings into a unified deep learning model for this task,which obtains a significant performance.Finally,A probabilistic matrix factorization framework with multiple constraints is constructed for personalized clothing recommendation.Previous works on collaborative filtering are not suitable for clothing recommendation,since they ignore that visual content is a essential factor for clothing recommendation.Additionly,user social information is another key factor for user decision making.To this end,we model these valuable clues and integrate them into a unified collaborative filtering framework.Extensive experiments on our dataset demonstrate the effectiveness of the proposed method for personalized clothing recommendation.