Computation Offloading Adaptive Middleware Of DNN Application In Edge Environment

With the development of artificial intelligence field,deep neural network(DNN)has become an emerging research hotspot.However,its computation-intensive and memory-intensive characteristics have expanded the problem of limited resources in mobile devices.Computation offloading is an effective way to improve performance and save energy consumption for devices.Some existing papers are aimed at deploying DNN applications in cloud services.And this approach,called the cloud-only,puts computational pressure on the datacenter and suffers significant data transmission overhead.Mobile edge computing(MEC)is a promising solution to assist mobile devices and cloud to offload DNNs.Due to the regional distribution of edge environments and the dynamics of mobile devices,supporting distributed DNN-based application on edge nodes should consider two aspects.On one hand,dynamic offloading needs to be carried out among available computing nodes,which are complex and dispersed.On the other hand,it is necessary to reappraise and make an effective offloading scheme according to current device context.In order to solve the above challenges,this paper proposes a computation offloading adaptive middleware,which supports dynamic offloading in MEC environment from the perspective of DNN program structure,and determines the optimal offloading scheme at the granularity of DNN layers through the cost evaluation model.This paper makes the following major contributions:(1)A design pattern of pipe-filter mechanism is proposed,which enables dynamic offloading in MEC environment.Firstly,according to the DNN program structure,the design pattern taking layer as filter and data transmission as pipeline is proposed.Secondly,source code is reconstructed according to the design pattern based on the static code analysis technique.Finally,the runtime mechanism ensures that the reconstructed DNN application supports dynamic offloading among devices,edge servers and cloud servers.In addition,it keeps applications available when device contexts switched or model structures changed.(2)An offline cost evaluation model is proposed,which automatically makes the optimal offloading scheme at the granularity of DNN layers.On one hand,the DNN structure,prediction of layer latency and network environment are modeled and introduced as the input of the cost evaluation model.On the other hand,the factors influencing the schemes are analyzed to balance the computation and transmission time,then the objective function of the evaluation model is constructed.(3)Three research questions are proposed,which evaluate the middleware of DNN application supporting the offloading mechanism and decision.The experimental results show that for a real-world DNN application,compared with the existing methods,the performance of proposed method can be improved by 3.8%to 66.6%,and the improvement is more significant with complex DNN models.In addition,the accuracy of evaluation model and the overhead of offloading mechanism are also in a reasonable range.