Research Of Big Data Platform Protection Based On Knox

Age of big data,data has become an important asset.However,the frequent events of data leakage impeded the development of big data.Big data platform is composed of many independent components and each component has its own mechanisms and safe schemes,moreover,the original design of big data platform did not considered the safe problems.All of these reasons caused many problems in applying the big data platforms.The complex schema makes difficulty in operating and the simple schema cannot protect the data comprehensively.How to protect the platform effectively,so that it can meet the needs of both operation and security anyway,that needs urge us to make researches of the protection of big data platform deeply.This paper mainly studied the safe technology of Hadoop platform which is the most popular big data platform in the industry.First of all,the paper researched the technology of safe protection of the Hadoop platform itself.After that,the article studied the four aspects of Hadoop platform,which included authentication,authorization,audits and data protection.The paper summarized the problems of Hadoop platform in the production environment in the internal knowledge of the platform,such as the complex mechanism of authentication,the decentralized mechanism of authorization and the insufficient of audit and data protection.After understood the problem of big data platform,the paper designed a comprehensive shame based on the technology of Knox to protect the big data platform which can meet the most requirements of many applications.The security schema proposed in this paper used the method of proxy to protect the platform,so that the requests of clients are operated by the gateway agent.Because all of the operation are carried by the proxy,the malicious clients cannot access the platform directly though that may has some vulnerabilities.The gateway can make centralized authorization and unified authentication,and it also can make perfect audit.Finally,the article designed and implemented a system which was named Security Monitoring Platform of Big Data based on the model of the paper.The safe model created in this paper adopted gateway to secure the big data platform,which is simpler model and easier management than the projects have realized in the production environment.The proxy server realized by plug-in can adjust its server dynamically according to different big data platform.In addition,the model implemented single sign-on at the gateway,which means a user can access any component of the internal platform only needs to authenticate once.The model also provided centralized authorization,including server layer authorization and fine-grained authorization.In the aspect of data protection,the model designed the operation of data desensitization which can protect the data of the platform effectively.In the end,some experiments were done to verify the model proposed in this paper.