Research On The Technologies Of Privacy Protection Of Verifiable Computation In Machine Learning

The rapid development of the Internet and the Internet of Things opens a new era of data and information,and the data has some new properties,such as volume,variety,velocity,value and vercity.As a way to realize artificial intelligence,machine learning focuses on how to obtain hidden,effective and understandable knowledge from massive data,and construct a data-driven reasoning and decision model,achieving the goal of "What is taken from the data is used for the data".Whereas,traditional machine learning algorithms contain some compute-intensive learning processes,thus there exist some application limitations for the resource-constrained users.Moreover,lacking of massive training datasets will lead to low performance and overfitting of the machine learning model.Fortunately,cloud-aided ma-chine learning technology has become a hot and cutting-edge research direction,and widely gained attentions from academia,industry and the government.Cloud computing is the de-velopment and evolution of many computer technologies,such as distributed computing,utility computing,parallel computing and virtualization,and users enable to enjoy the end-less storage and computing resources on the cloud platform in a pay-per-use manner.That is to say,with the assistance of the cloud server,users conduct model training and optimiza-tion,which not only greatly reduces the computing cost and maintenance cost of the client,but also realizes the effective utilization of the distributed datasets.Nevertheless,since that users' data often contain some sensitive information,the cloud servers are usually semi-trusted,which inevitably brings some security problems in cloud-based machine learning.Firstly,data outsourcing makes users lose physical control over them.How to ensure the pri-vacy of training dataset and the verifiability of the calculated results during training process has become one of the challenges.Secondly,the prediction model obtained from the train-ing process can be regarded as the user's intellectual property.How to realize the privacy protection of model parameters in the prediction stage is also one of the problems.Thirdly,prediction samples usually contain personal privacy information.How to ensure the priva-cy of prediction samples and its results is also one of the challenges to deal with.Finally,federated machine learning is an efficient and collaborative learning paradigm with multiple users participation,and the group user dynamic updating occurs frequently.How to realize the data privacy protection of group user in dynamic scenario is one of the challenges.In recent years,plenty of experts and scholars have devoted considerable efforts to the problem of verifiable computation technologies in machine learning.However,the existing schemes still suffer from the following security and privacy problems.Firstly,matrix mul-tiplication is the basic operation of machine learning algorithm and the existing large-scale matrix multiplication schemes cannot achieve the requirements of users' data privacy pro-tection and the public verifiability of calculated results simultaneously.Secondly,in terms of Single-Layer Perceptron training and prediction stages,the existing schemes cannot achieve the privacy protection of data during the whole process.Finally,the existing schemes can-not support group users to update dynamically in federated machine learning.To overcome these problems,in this dissertation,we focus on the key technologies of privacy protection of verifiable computation in machine learning,the main contributions are summarized as follows:1.We propose a publicly verifiable computation for batch matrix multiplication.Differ-ent from the traditional outsourcing matrix multiplication algorithms,our computation task is to conduct two matrices multiplication operation,where one is public and con-stant and another one is provided by group users.Based on the privacy-preserving matrix transformation technique,the proposed scheme solves the problem of users'data leakage.In addition,exploiting the matrix digest technique,two-dimensional matrices can be transformed into one-dimensional vectors.Therefore,it can dramat-ically reduce the computation overhead both in the first stage key generation and the third stage matrices multiplication,dramatically improving its efficiency.(Chapter 3,the main work has been published at Information Sciences,479,664-678,2019)2.We propose a novel Single-Layer Perceptron training scheme which can derive mul-tiple predictive models for different patterns simultaneously.It solves the issue of the data privacy leakage in training process.Meanwhile,we introduce the verification mechanism into our scheme.If the cloud server cheats the client by returning an in-correct value,the dishonest behavior will be detected with the probability of 100%Besides,based on the secure two-party computation,we design a lightweight privacy-preserving prediction algorithm.In predictive stage,the predictive model,the new data record and its final classification result can be well protected.(Chapter 4,the main work has been published at Soft Computing,22(23),7719-7732,2018)3.We propose two privacy-preserving and asynchronous deep learning schemes.Group users hold different encryption keys in the first scheme.Leveraging proxy re-encryption technique makes the model parameters be aggregated and updated on the server side This method can preserve each participant's input privacy and provide dynamic update secrecy,i.e.,forward and backward secrecy inherently.We adopt group key manage-ment method to overcome the challenge of data privacy leakage of each participant and model parameters.A common encryption key is shared among a group of user-s and will be updated once every user revokes.Security analysis demonstrates that the second scheme has the property of backward secrecy.(Chapter 5,the main work has been published at IEEE Transactions on Industrial Informatics,DOI:10.1109/TI-I.2019.2941244)4.We design a privacy-preserving and verifiable online crowdsourcing scheme,which can protect the input data privacy of group workers.In different classification tasks,our scheme supports worker dynamic adding and revocation with a minimum com-putation overhead.Moreover,we design a probabilistic verification mechanism to identify the returned results.In aggregation stage,since no decryption operations are involved in our scheme,it is lightweight and efficient.(Chapter 6,the main work has been submitted at IEEE Transactions on Dependable and Secure Computing)...