Robust And Communication Efficient Multi-Model Solution For Cross-Silo Federated Learning

Over the past couple of decades,deep learning stands on artificial intelligence(AI)has revolutionized a broad array of fields such as healthcare,transportation,and,manufacturing etc.,mainly attributed to the advancement of computer vision and natural language processing tasks.Nonetheless,the main ingredient for executing these tasks requires an immense amount of data collected from dispersed participants largely in privacy-invasive ways.The recent formation of different data regulations pertaining to privacy has contributed to the development of a privacy-preserving machine learning paradigm called federated learning.Federated learning is a collaborative machine learning approach that enables to train a single shared model without sacrificing data privacy.Instead of accumulating data to a central point,participants are allowed to collaboratively train a machine learning model by only uploading model parameters to the coordinating server for global aggregation.This newly emerged approach has two inherent properties of privacy-preservation and distributive computation,but absence of integrating the idea of an independent personalized model for each participant limits its application in practice,which is important in cross-silo settings particularly in the fields of healthcare or e-commerce due to different training objectives and heterogeneous nature of tasks.First,this thesis proposes a novel training framework for personalized federated learning,more precisely a secure and multi-model heterogenous federated learning.The proposed framework enables each participant in federated network to design an individual personalized model with heterogenous architecture,unlike the standard federated learning approach which relies on a unified model architecture for all participants without any personalization.Furthermore,the proposed framework adopts differential privacy technique to ensure that the exchange of model information between participants and the coordinating server with minimal sacrifice of individual model utility.To validate the performance of the proposed approach,extensive experiments on MNIST and CIFAR benchmark datasets are conducted using heterogeneous model architectures.The results obtained from this study shows that the proposed framework achieved stable performance and outperforms the baseline algorithms.Second,this thesis introduces a collaborative knowledge transfer training framework for a cross-silo setting with only one round of communication.The proposed design efficiently removes the constraint of iterative-based approach which is considered a bottleneck for crosssilo federated learning.Moreover,the framework offers the flexibility to all participants in the federated network to design any classification model from lightweight to moderately complex architecture.To validate the performance of the proposed framework this study performed extensive experiments on benchmark datasets and the results obtained show that the proposed framework achieved slightly better or comparable performance against the baseline algorithms in only one round of communication.