Research And Application Of Distributed Neural Network Algorithm Based On Domain Adaptation

In recent years,deep neural networks have been gradually applied in various fields with great success.With the dramatic increase in the number of terminal devices and Internet of Things devices,these devices are attractive targets for machine learning applications because they are usually directly connected to sensors that capture large amounts of data streams.With the distributed computing architecture,the distributed neural network composed of cloud,edge and terminal devices has significant advantages in the application scenarios of the Internet of Things.The idea of distributed neural network is to map the trained global deep neural network model to a distributed device hierarchically.The network has multiple outlet nodes.If the exit confidence is high,the network can exit directly from the local terminal device,which reduces the use of computing resources and communication costs.However,distributed neural networks face great challenges in network training and formal deployment because of their architectural features and distributed nature.Distributed neural networks cannot process data with inconsistent distribution when deployed.The data in traditional distributed neural network is obtained by each terminal,and can only be obtained from the actual running process.Under hierarchical architecture,each terminal device can only get local data,but not all knowledge,which makes the training of the model more difficult.For the above questions,the main work of this thesis is as follows:1.Unsupervised domain adaptation can significantly improve the classification performance of unlabeled target domain,in this thesis,we propose TMAAN an adversarial domain adaptation algorithm based on a meta optimization strategy and taskoriented feature alignment,the algorithm actively helps domain alignment in classification task by performing feature decomposition and alignment.In addition,the algorithm solved the optimization inconsistency between domain alignment task and classification task.Our experiments show that the algorithm TMAAN has better performance than other classical algorithms.2.In order to solve the problem that the traditional distributed neural network is difficult to handle the data with large distribution differences when deployed,a domainadaptive distributed neural network architecture DA-DDNN is proposed.In addition,due to the distributed network architecture,it is difficult for the network at the terminal device layer to learn all the knowledge.A collaborative optimization method based on edge nodes is proposed to enable the terminal to converge to the edge after initial feature extraction,and then train again and synchronously update the device layer parameters.Most samples can exit locally while maintaining high overall and local exit accuracy.Finally,the excellent domain adaptability and classification performance of DA-DDNN architecture are verified in experiments.3.Based on the domain adaptive algorithm and the distributed neural network model,a domain adaptive distributed neural network image classification system is designed and implemented in this thesis.Domain adaptive based distributed neural network models are loaded into the system after pretraining.The system provides three different network topologies and has basic functions such as data preprocessing,model inference,image classification and visualization results display.Finally,the function and performance of the system are tested.