Design And Implementation Of Multi-Devices Collaborative Distributed Neural Network System For Video Streaming

It has become an indispensable part of people’s daily life to develop video applications on smart devices(such as mobile phones,and tablets)and resourceconstrained devices with sensing capabilities(such as cameras and wearable devices).With the development of the 5G mobile communication network and edge computing technologies,computational offloading inference technology has been developed significantly.Through computational offloading technology,computational storage resources are deployed on different arithmetic devices such as resource-constrained terminals,intelligent assistance terminals,and edge servers which close to end-users.This scheme can solve the following problem.Owing to low computational performance,few storage resources,limited energy consumption,and battery endurance,the pass-through terminal devices cannot independently and efficiently perform computational task processing.However,DNN model partitioning is only applicable to pictures but not video streaming analysis,due to its high single-frame processing latency,high network environment requirements,and high offload scheduling overhead.Early exit does not apply to real-time video streaming analysis because the branch discrimination criteria affects accuracy.It is challenging to meet the latency requirements for superficial branches of complex tasks.This thesis investigates the design and implementation techniques of a distributed neural network system for multi-terminal collaboration of video streams,with the following main works.(1)A content-aware multi-terminal distributed neural network inference framework is proposed.The entropy value of contextual video frame information quantity is proposed as a parameter of the input state value in the offloading inference algorithm.It can quickly and adaptively offloads a video frame processing task in a video stream sequence to the corresponding path.The computing resource device with optimal efficiency by specifying the action space and computing device correspondence to speed up the offloading.The speed and accuracy of decision making optimize the time delay and energy consumption generated during video frame analysis and processing.(2)In the content-aware multi-terminal distributed neural network inference framework,the terminals with different computing power is researched on.,Based on semantic richness,a lightweight video frame grading algorithm is proposed to optimize the offloading inference strategy by grading the computational processing tasks.It can efficiently utilize the computational resources of local and edge devices.YOLOv5s is proposed as the backbone network structure to design lightweight convolutional neural networks.It can speed up the computation by effectively separating important feature information and reducing model parameters.(3)Based on the above two algorithms,this thesis designs and implements a multi-terminal distributed neural network system for video streaming.It evaluates the amount of video frame feature information at the resource-constrained terminal side,and offloads the video frame computation task in a reasonable and optimized way.The method is combining bandwidth and load of each device for dynamic decision scheduling based on the end-edge network architecture.The simulation verifies that the proposed algorithm model is oriented to real-time.So it is suitable for real-time video analysis task offloading reasoning applications.