Research On Monitoring Module Of Virtual Screening Based On Hadoop

Virtual screening is an important means of drug discovery with huge amounts of data involved in large-scale virtual screening.Traditional virtual screening need to prepare large amounts of data,such as query molecule compounds,upload data,and download results in which chemists workload is very complex and cumbersome.The automation of molecular docking not only can reduce the burden of the chemist,but also can improve the efficiency of the parallel docking.The characteristics of the distributed programming framework map reduce running in parallel with massive data processing capacity provided by hadoop enable it to be a very effective platform for implement automation of virtual screening in distributed environment.The techniques of resource monitoring and job scheduling provided by hadoop are extended to enable them to better meet applications of virtual screening which is a very important function in virtual screening automation system.The main research work of the monitoring module in virtual screening is mainly divided into the following two aspects:Firstly,according to the molecular docking job can be executed in parallel with its long executive time in large difference,this paper chooses and extends fair scheduler provided by hadoop for scheduling job queue and uses modulo N algorithm combining with custom hash partition algorithm to divide job into data blocks to achieve load balancing;Secondly,the monitoring module use open interfaces provided by hadoop which handle logging information to obtain load information and executive information.It also provides real-time page to display information of system details to users.Research work of this paper can help users to understand the system operation and the progress of job.