Design And Implementation Of Merchants Portrait System Based On Feature Label

With the rise of "recommended" traffic,user portrait system has gradually become a hot research area in recent years.The emergence of Alibaba’s Dharma disk,Baidu’s Sinan,Tencent’s social big data platform and other systems represents that "user portrait" has gradually become one of the necessary capabilities of an Internet enterprise.However,with the rapid development of user portrait system,the traditional user portrait system has exposed many disadvantages,such as unfocused business,poor timeliness and classification granularity is too large.In this thesis,we design and implement a business portrait system based on feature tags,which solves the difficulties of accurate access,scattered data,frequent business changes,and difficult classification of businesses.In this thesis,the merchant portrait system is divided into two parts:feature subsystem and label subsystem.Through the feature subsystem,data migration,processing,storage and other functions are realized.Through the label subsystem,merchant circle selection,merchant judgment,label calculation and other functions are realized.The main work of this thesis includes:(1)This thesis investigates the development status and potential problems of the user portrait system,analyzes the actual business pain points of the company,and expounds how to solve the existing problems of the business portrait system based on the feature tag combined with the actual business of the company.(2)The storage design of the system is carried out,and the features of the core data of the system are analyzed.Offline feature data has the characteristics of discreteness and large amount of data,so column database HBase is selected as the storage and optimize the design of row key.Real-time feature data has the characteristics of high real-time requirement and frequent reading and writing,so Redis cache is selected as the storage.Tag data has the characteristics of invariance and high query timeliness,so elastic search is selected as the storage of tag data after calculation.(3)The core functions of the system are designed and implemented,focusing on feature data migration and tag calculation.Feature data migration uses spark SQL to migrate feature data from Hive table of business to HBase cluster in parallel,and pre allocates region to avoid data skew.Label computing realizes label parallel computing in memory by slicing,controls the parallelism by Redis based distributed lock,and precompiles the expression by using Aviator evaluator to speed up the computing efficiency.The research results show that the business portrait system based on feature tag is a user portrait system focusing on business attributes,which can solve the problems of data processing and business classification in the process of business operation.After the business portrait system is online,it successfully saves the cost of enterprise management and improves the value of individual businesses.