Key Technologies For Blockchain-based Collaboration In Multi-cloud Context

In the early stage of cloud computing,users heavily depend on a single cloud service provider.As the number of cloud service providers continues to grow,the demand for multi-cloud context becomes more pronounced as users aim to reduce dependence.However,due to business competition among cloud service providers,it is challenging to achieve collaboration.Blockchain technology provides strong technical support for research on collaboration issues.In the early days,blockchain is primarily used as an underlying technology for cryptocurrencies.The invention of smart contract technology enables the execution of complex operational logic and expands its range of applications.Nowadays blockchain is regarded as an underlying network that provides a foundation of “trust” and is widely used in scenarios that require multi-party collaboration.The decentralized nature of blockchain enables collaboration without relying on third parties.The immutability of on-chain data guarantees reliable records of the behaviors of participants.The support of smart contracts enables programmability and automation of the collaboration process.Such blockchain-based collaboration brings new solutions to traditional collaboration problems in the multi-cloud context.However,there are a few critical challenges.Collaboration can be divided into three layers,i.e.resource layer,data layer and service layer.Collaboration in different layers faces different challenges.First,collaboration in the resource layer cares more about efficiency.For instance,multiple cloud service providers cooperate to share or exchange cloud resources.It is important to guarantee the economic efficiency of the collaboration mechanism since different cloud service providers have different interests.Second,collaboration in the data layer cares more about privacy.For instance,multiple parties cooperate to share data.It is important to prevent the leakage of sensitive data.Especially,the transparency nature of blockchain makes this problem more pronounced.Third,collaboration in the service layer cares more about security and performance.For instance,in blockchain-based federated learning,participants can perform attacks easily without the supervision of a third party.Moreover,blockchain introduces additional time and storage overhead to collaboration.This dissertation is based on a general collaboration model called “Cloud Alliance”proposed by early researchers.It takes collaborative federated learning in the multi-cloud context as the typical scenario and conducts research on relevant key technologies,including the three layers and four challenges mentioned above.1.Efficient collaboration mechanism based on blockchain: in response to the issues of inadequate resource utilization and scarcity of device resources,within the cloud alliance,multiple clouds need to collaborate to exchange and optimize resource allocation.The participants of collaborative tasks such as devices in federated learning might need to buy additional resources to address the shortage.This dissertation models this scenario as a multi-unit double combinatorial auction problem and designs a decentralized auction mechanism based on blockchain.It proposes a greedy allocation algorithm that demonstrates near-optimal economic efficiency.Furthermore,a pricing algorithm based on the second price and user feedback is introduced to motivate participants to provide high-quality services while limiting their ability to obtain excessive profits through price adjustments.Such behaviors may lead to price instability and impact economic efficiency.The immutability of transaction records on the blockchain guarantees the fairness and security of user feedback.2.Privacy-preserving collaboration mechanism based on blockchain: in the cloud alliance,cloud service providers can record the behaviors of collaboration participants and evaluate their reputations.A global comprehensive view of each participant will be enabled by sharing these reputations over the blockchain.However,the business connection between cloud service providers and their connected devices or consumers should remain confidential.This dissertation takes federated learning as a typical service and designs a blockchain to share the reputation and preserve privacy.An algorithm based on the restricted subjective logic model is proposed to evaluate the reputation of each participant.A shuffle-and-pair algorithm is proposed to hide the connection between cloud service providers and devices.A double-check method is proposed to guarantee the integrity of reputations before being included in the blockchain.3.Secure collaboration mechanism based on blockchain: services provided by Cloud Alliance are closely related to user business and data.Therefore,higher security requirements for collaboration arise.This dissertation investigates a typical service,i.e.collaborative federated learning and designs a secure blockchain-based federated learning mechanism.This dissertation guarantees collaboration security by addressing the device selection problem in federated learning.The contribution of each device is evaluated using Shapley values.An exploration-exploitation algorithm based on device contribution is designed to gradually eliminate potential attackers.However,the computation of Shapley values incurs high time overhead.Thus,a reliable parallel computation algorithm is designed based on blockchain sharding,reducing system time overhead.4.High-performance collaboration mechanism based on blockchain: providing largescale collaborative services faces two significant challenges,i.e.latency and storage.Take federated learning as an example.Compared to centralized federated learning systems,the usage of blockchain incurs additional time overhead.As blockchain grows,storage overhead increases continuously.This dissertation proposes a lightweight blockchain for federated learning.First,a periodic refreshing storage policy is designed based on erasure coding.This policy fully utilizes the limited storage space of lightweight devices to guarantee storage reliability while reducing storage overhead.Second,a novel consensus algorithm named Proof of Reliability is proposed and a most-stake aggregation algorithm is designed.Both algorithms reduce the impact of unreliable devices and decrease system latency.