Design And Implementation Of Edge-device Cooperating Dnn Inference Accelerating Method

In the industrial Internet environment,deep neural network applications have been widely used in industrial production.These applications are usually deployed in smart terminal device and have the characteristics of large amounts of calculation and data,while requiring low inference latency.However,the inference latency of device is high due to the weak performance of the processors in the existing smart terminal devices,and the method of sending original data to the cloud computing center for inference has the risk of privacy leakage.In view of the shortcomings of existing computing methods,edge-device cooperating inference can reduce the inference latency in device by comprehensively utilizing the computing capabilities of device and edge.However,the performance of the heterogeneous processors in smart terminal device are different,and the high temperature in the industrial production environment limits the performance of the processors,at the same time,the environment between device and edge is complex and changeable.This situation brings challenges to building edge-device cooperating inference system.In response to the above problems,existing works have made research on technologies such as DNN parallel computing,device computing acceleration,and edge-device cooperating inference framework,but the existing works still have the following problems: On the one hand,DNN inference acceleration technology for device haven't considered the temperature constraint to processors in device,but divides the tasks in a coarse-grained way,which haven't pay attention to the different structures of the layers in DNN and the difference of heterogeneous processors,therefore haven't fully utilized the computing ability of the device.On the other hand,the edge-device cooperating inference methods haven't considered the computing ability of the device and edge,the energy consumption,and the network conditions,which makes it difficult to ensure low inference latency and stable operation of devices.To solve this problem,basing on the characteristic of strict latency requirement in device DNN inference,this thesis proposes a edge-device cooperating DNN inference accelerating method.First of all,this thesis proposes a heterogeneous processor inference acceleration technology for smart terminal device under temperature constraints.The technology is divided into two parts: device processor frequency dynamic setting and deep neural network single-layer calculation load division.The former one can set the working frequency of the processor in the device according to constraints such as ambient temperature and processor performance,so as to ensure the stability of the terminal device and maximize the computing power of the terminal device; the latter one can divide the calculation load of the single layer in the neural network according to the layer's structure and the computing power of the heterogeneous processors,thereby reducing the inference latency of the device.Secondly,this thesis proposes an edge-device cooperating inference mechanism under multiple constraints.The mechanism is divided into two stages,the edge-device cooperating model segmentation stage and the online dynamic adjustment stage.The former one can set the division position of the DNN under the constraints of environmental conditions to reduce the latency of inference; the latter one can dynamically adjust the model division position when environment changes,thereby ensuring the robustness of the edge-device cooperating inference mechanism.Finally,according to the theoretical research results,this thesis designs a deep neural network inference system based on edge-device cooperation,and through multiple sets of comparative experiments proves that the mechanism proposed in this thesis improves the computing performance of smart terminal devices and reduces the edge-device cooperating inference latency.In summary,this thesis is oriented to the application of deep neural network in the industrial internet environment,designs a deep neural network inference acceleration technology based on edge-device cooperation,and builds a corresponding prototype system.Experiments show that the mechanism proposed in this thesis can effectively reduce the latency of deep neural network inference,and ensure the stable operation of edge-device collaborative inference.