Research On And Application Of Message Queue Model Under High Concurrency

In addition to increasing the number of servers or configuring a server cluster to achieve scalable architecture design to alleviate high concurrency, adopting asynchronous message processing through message queues is also essential to the solution. A message queue separates the message sender and receiver in an asynchronous way, which can guarantee the transmission of the message efficiently and realize the decoupling between modules. In the context of the cloud computing and big data, the traditional message queuing failed to consider the message itself, it makes the high performance deployment and user customization of the message queuing service based on the attributes of the message very difficult. At the same time, a variety of complex application scenarios requires message queues can flexibly change their deployments, while the traditional message queues are generally designed for a specific application development, so when the development plan changes it is difficult to realize the dynamic extension. Based on these two problems, this thesis mainly studies the high concurrent message queue model which can be configured under the Multi-application, Multi-tenant, and Multi-factor(3M) configurable data platform. The main research work presented in this thesis is as follows:Firstly, it analyzes the characteristics of a data processing platform used in the mobile Internet and big data background, and puts forward a kind of application structure based on the 3M data platform. In this architecture, the 3M style applications raises new requirements for the data processing platform, and these new requirements point out that the traditional message queue model is difficult to implement on-demand allocation according to the requirements of different applications, and cannot guarantee the core business performance. A virtual message queue group is proposed to realize the isolation among multiple mode message queuing, so that message queue can deploy multiple message queue service according to application requirements.Secondly, it investigates the traditional message queue model, and points out the limitations of the traditional message queue model in the design. Traditional message queuing services are generally developed for a certain application, their configuration can only applied to the configurable parameters related to message queue itself, and could not configure the property that was originally carried by the message. Considering the requirements needed by 3M data platform, a High Concurrent Message Queue Model(HCMQM) is proposed. In HCMQM, the isolation between a variety of mode message queue is achieved through the virtual message queue group, and the flexible management of message queue services through the configuration file.Finally, it builds the technical platform to showcase the process flow of HCMQM usage for some specific applications, and determines the configuration file template through analysis and comparison. On this basis, the development of the message queue model is completed. The simulation of a wide variety of high-concurrent message queue combination are carried out to verify the feasibility of the model, and it shows that based on the message itself a variety of flexible deployment plan can be achieved through the configuration file.