Research On Large Scale Programming Online Judge System

Programming online judge is a way to fit the large amounts of paticipants with the limited programming educational resources, and it makes large scale programming assessment possible, and becomes a solid fundation of online programming teaching. The widely spreaded programming online judge systems are mainly used for ACM-ICPC(Association for Computing Machinery-International Collegiate Programming Contest),and praticed in algorithm training at the very begining. These systems test submitted code with black blox testing model, and give points according to the number of test cases the answer passed. Nowadays, lots of universities have brought ACM-like online judge systems into programming courses. However, there are some drawbacks of these systems. First of all, with the revelution of programming languages and software development technology,the black box testing model can not meet all the teaching requirements of newly invented programming languages. For example, black box testing can tell the wrong behaviours of the submitted code, but has no way to judge for object-oriented programming philosophy.Secondly, the contest rule of ACM punish the unpassed submission, and it breaches the philosophy of test driven learning. It also avoids the high submission concurrency of large scales examinations. Lastly, the real name registration environment allows the security to be less considered. And some safe mechanisms have maintenance and migration over heat.This thesis discusses the status of programming online judge systems, and analyzes the needs of programming examinations. Widely used dedicated judge systems for programming competitions like ACM has disadvantages. And there are needs for online course judging, such as system scale, real time judging, security, judge fineness, and multi-grading points judging. According to these reasons, this thesis proposes a multi grading points judge scheme which is based on distributed computing, virtualization, and plugin mechanism. This thesis also designs and implements a programming judge system with online scaling enabled. By using the queue provided by Active MQ, the judge system implements a producer-consumer module to meet the need of online scaling and fault tolerance in a large scale examination. This module supports adjusting the amounts of judgers online according to the system load, and it allows some judgers going offlineaccidentally. By using message queue, cache, and distributed computing, the system can reach a high throughput, and serve a large amounts of users at the same time.This thesis is mainly focusing on the following aspects.(1) In respect of security, this thesis studies the safe sandbox technology, and reinforce security of the judge system with half and full virtualization mechanisms. The safety architecture which is formed by operating system based user and file system permission mechanism, set UID and ulimit sandbox, LXC(Linux Container), and virtualization technology, which makes the architecture capable for both ensurance of a safe and clean judging environment, and running at a better performance and management cost.(2) In respect of performance, this thesis also optimizes the performance of disk IO(Input/Output) and the start up time of judgers to gain a better throughput.(3) In respect of diversified judge requirements, the thesis implements a plugin mechanism, and meets the needs of online classes, such as multiple grading points and judge fineness by using gray box testing model.Results of experiments show that the judgers and its supported system are capable for scaling and fault tolerance, which means judgers can be online or offline at any time with no impact on correctness of the system. As a result, online scaling can be done by simply adding judgers, which makes a large scale programming judging possible.