Design And Implementation Of Container-based Algorithm Development System

With the arrival of the era of big data,the research on algorithm attracted much attention.There are more and more tools and open source technologies for algorithm research,but there is no universal and functional platform for algorithm development.Based on the above background,this paper introduces a container-based algorithm development system,it provides a relatively complete solution for the development and deployment of the algorithm.From the point of users who develop algorithms,the paper analyses and introduces the system.Using Kubernetes,Ceph,Gitlab and SSM framework technology,the container-based algorithm development system is developed.The system mainly includes six functional modules,user management module includes user self-management,team management and other functions;data set management includes data set upload,sharing and basic information management;model management realizes model upload,model information management,model publishing and sharing;algorithm management provides debugging,editing,management and sharing of algorithms.The experiment result tracing provides the functions of result management,tracing and visual comparison.Task monitoring realizes the function of monitoring the resource consumption when algorithm is running,including memory,CPU,storage and so on.The core functions of the system include algorithm editing and debugging,algorithm fast publishing,result tracking management and algorithm resource consumption monitoring.It provides an isolated algorithm development environment,supports the dynamic expansion of algorithm computing resources,and provides a reliable and efficient algorithm development platform for users.The system has been tested and used online.The system can support the user's needs for algorithm debugging,publishing algorithm and results tracking,and make users develop algorithms easily.