| In recent decade,the problem of food safety has become a hot topic concerned by the society.The talk about frequent occurrence of food safety has the characteristics of fast propagation velocity and far-reaching influence.So,the requirements about timeliness of topic detection get higher than before.At present,the world has stepped into the era of big data,and the amount of data is increasing exponentially.The explosion of data makes the traditional framework not process issues in real-time,the research of food safety network on public opinion is imminent.The main research contents of this paper include the topic extraction based on the Strom distributed framework,the text feature extraction based on the neural network,and the improvement of the distributed food safety network public opinion system.The main work is as follows:1.At present,the topic detection has essential requirements in time,therefore,this paper has improved Single-Pass topic detection algorithm based on Storm distributed framework.The algorithm improves the speed of the topic detection effectively.The empirical results show that with the increasing of the number of nodes in the cluster,the running speed of the Single-Pass algorithm is raised linearly,under the case of keeping the clustering accuracy.As a result,the efficiency of algorithm can be remarkabley increased.2.Aiming at the low precision of Single-Pass algorithm,this paper proposes the Single-Pass algorithm based on autoencoder network.This algorithm used multiple hidden layers for data reconstruction,in order to extract the characteristic of the original data,reduce false detection rate and improve the clustering accuracy.In additional,This algorithm also proposed the concept of edge text,through the recalculation of the edge text to solve the problem caused by data sequence,further improving the accuracy of text clustering.3.According to the characteristics of text data about the field of food safety,this paper improved the Storm distributed topic detection algorithm that text data is sorted first through the establishment of food category decision tree and then clustered,further improving the timeliness of the topic detection and the efficiency of the system effectively. |
