Design And Implementation Of An Image Processing Cluster System For High Concurrent Scenes

Due to the demand of market,the image processing technique has received great attention in recent years,and the relative applications are developing now.Various Internet and traditional software vendors are launching many products of online image processing services,such as vehicle license plate recognition service in traffic system,the service of real time sexy recognization in live broadcast industry and online landmark identification service,etc.Because the providers of these services often have large and active customer groups,there are high requirements for the throughput,real-time and stability of the online image processing service.In response to the requirements of the above image processing services,this paper presents a cluster system based on Java technology stack,which uses the RPC idea and soft load balancing technology.The cluster system is designed for the provider of cloud service,they can deploy the cluster system in their own hardware system,to allow third party developers or themselves to deploy image processing algorithm services to cluster system.Finally,these services will be available to users.This cluster system uses the Zookeeper framework to implement a configuration center,providing the system with the consistency of each node and the request of the Domain Name Resolution Service.In the configuration center,the user's request can be effectively shunted to adapt to the high concurrency scene by implementing a soft load balancing algorithm.After the request is shunted through the load balancing service,the caller will establish a single to single connection with a cluster node,and begin to transmit the image data,and call the specified image processing algorithm service to process.In this process,the cluster system relies on a high-performance RPC framework based on Netty and Protobuf to improve the throughput of single server to client request,and ultimately achieve the goal of improving the overall performance of the system.Besides optimizing the performance,this system also uses the Class Loader mechanism of Java to allow users to upload the code dynamically,and achieve the purpose of realizing the hot loading function of the algorithm service without the need to turn off the system service.In order to improve the usability of the image processing cluster system,it has a more practical application significance.