Research On Message Oriented Middleware Technology Based On Data Content

With the in-depth development of the information technology revolution on a global scale,the widespread of Internet software and hardware,and the rise of the "Internet +" industry form,the era of big data has arrived.The huge increase in data volume provides strong support for data mining technology,bringing more intelligent and convenient network services to human society,on the flip side,the network transmission and storage requirements put forward by it have also raised a severe challenge to existing software technologies,spawning the continuous evolution of distributed technology.The messageoriented middleware technology is a software technology in a distributed system that provides efficient data distribution services for both ends of communication and supports distributed system to smoothly respond to large amounts of concurrent access requests.Nowadays,under the growing network data scale,the bottleneck of message-oriented middleware's performance in term of throughput and latency while facing data floods peak has become much more prominent.The huge data transmission demand make data service providers be obliged to deploy a large number of servers to form complex middleware clusters,which lead to a soaring cost of cluster operation and maintenance.Not only that,network bandwidth and storage resources occupied by data transmission exerts negative influence on overall service efficiency of the distributed system.Existing message-oriented middleware improves system performance by traditional software optimization methods such as optimizing IO,reducing redundant processes,increasing the number of concurrent threads,but with the evolution of message-oriented middleware,the performance improvement brought by such method has encountered a bottleneck.Message-oriented middleware urgently needs a means to further improve performance.To solve this problem,this thesis puts forward the idea of data-driven message transmission,and in view of the reality that distributed computer systems urgently need a message-oriented middleware with higher performance,combining with the burgeoning machine learning technology,a message-oriented middleware technology based on data content is proposed.Aiming at the throughput bottleneck of message-oriented middleware,in order to improve the current situation of message-oriented middleware's high occupancy of bandwidth and storage resources,this thesis designed an autoencoder training module and innovatively introduced autoencoder into message-oriented middleware to minimize compression ratio of numerical data,and thus improve system throughput rate.In response to the low latency demand of critical hotspot data in distributed system,this thesis proposes a hierarchical message transmission that allows the message priority to be set by parsing the message content.By storing messages with high priority basing on RAM and utilizing memory map technology to optimize the disk storage process of ordinary messages,prioritized high-speed transmission of critical data can be achieved,which is able to provide support for rapid response of important application services.On this basis,this thesis designed a messageoriented middleware system for distributed environment.Based on the design schema,this thesis completes the implementation and deployment of the overall system.Finally,this thesis performs performance testing and usability testing on the system.The test results show that the message-oriented middleware designed in this thesis has advantages in latency performance of a part of messages and throughput performance.This result shows that it is feasible to improve performance of message-oriented middleware by introducing data analysis methods,which provides new ideas for the optimization of message transmission performance and is of great significance to the research on distributed message transmission.