Analysis And Implementation Of Text Mining For Call Centers With Big Data

With the rapid development of call center business and technology, each call center has accumulated a large number of call records with audio and text format, forming huge data assets. But it is difficult to analyze this unstructured data effectively, so how to mining the valuable information from the call records has been a problem for academic and business research.In this paper, we guided by the project requirements, uses the research method of integrating theory with practice, including domestic and foreign research situation of text mining, data preprocessing for call center’s data, automatic text categorization and topic mining. First of all, based on the existing technology and research in the field, the paper put forward the complement or innovation points in text mining for call center. Secondly, we aim at the characteristic of the data set for call center; design a set of data preprocessing schema for cleaning and pre-treatment word for real data. After pretreatment, this data can be used for follow text mining algorithms. Then, in order to solve the skew and noise problem of call center data sets, we proposed an improved automatic text classification algorithm for classifying data and an A-LDA topic model based on associated attributes for topic mining. Meanwhile, for the massive data, in order to improve the performance of text analysis algorithms, we achieve MapReduce distributed computing for above two algorithms. Finally, on the basis of earlier algorithms, we designed and implemented a system that integrated the algorithms implementation and show the mining result in real-time, fast and friendly ways by the form of the webpage diagrams.In summary, this paper based on the research of text mining technology in the call center, design and implement a big text data analysis system that contains the data preprocessing, distributed text mining algorithms and visual presentation, used for reference of call center decision-making.