Design And Implementation Of FAQ Question And Answer System Based On Deep Learning

In recent years,with the rise of e-commerce platforms and the needs of customers,service providers have begun to form their own customer service teams to answer questions raised by customers.Usually,the questions asked by many users are similar,which brings a lot of repetitive work to the customer service team.Therefore,collecting common questions and standard answers in advance and quickly returning answers to common questions to users can greatly reduce customer service the work intensity of the team,and reduce the labor cost of the enterprise.Considering that Frequently Asked Question(FAQ)data sets are usually organized in a hierarchical directory structure,that is,it has the characteristics of dividing according to the topic.This paper uses the BERT topic classification model for FAQ semantic representation feature learning,and the output vector of the penultimate layer in the model coding layer as the semantic feature representation vector of the FAQ text.This paper crawled the FAQ data sets of multiple platforms such as Jingdong Mall and conducted comparative experiments.The experiments showed that the semantic representation feature learning effect of the BERT topic classification model is better than the BM25 algorithm and the word2 vec model.In order to speed up the search rate of question-answer pairs,the FAISS vector search engine is used to build an index for the FAQ question-answer database,and the IVFPQ index uses the inverted file system and product quantization technology to speed up vector retrieval.Through vector retrieval,the Top-K question-answer pairs that are most similar in semantics to the user input question are obtained.This paper also uses the BERT model to predict these K answers as the probability of the next sentence of the user input question,and Re-Rank the K answers,and finally return the best answer to the user.On two GTX TITAN X GPU environments,an index construction and retrieval experiment was carried out on 10,000 data sets,and it only takes about 0.3 seconds to retrieve the vector from the index file.This paper designs and implements a FAQ question and answer system based on deep learning and FAISS vector search engine,including five modules: semantic feature extraction,index construction,semantic retrieval,question matching and answer Re-Rank.By gradually expanding the data in the FAQ question-answer database from 10,000 to150,000,the performance of the system was tested.The results show that the FAQ question and answer system implemented in this paper can return answers to users in about 1 second.