Research On Task Offloading Mechanism Of Vehicular Network Based On Deep Reinforcement Learning

The emergence of cloud computing has brought the vehicular networks into a stage of rapid development.In the meantime,many emerging vehicular applications,such as autonomous driving and vehicular augmented reality,have brought exponential data growth and placed higher computation,communication,and storage resource requirements on vehicles.Mobile Edge Computing(MEC)is a promising solution to these problems,which provides rich computation,communication and storage resources for vehicular applications at the network edge close to the vehicle,effectively reducing the execution time of applications.However,due to the inherent characteristics of vehicular network,such as mobility of vehicle,dynamic network topology,and uneven distribution of vehicle density,the user’s service experience is still influenced by unpredictable network congestion and uneven task load.In addition,different vehicular tasks have different resource requirements.A simple strategy can’t achieve reasonable offloading for all tasks.It is necessary to design a corresponding task offloading algorithm for specific scenarios or propose a task offloading algorithm that can quickly adapt to different scenarios to reasonably decide whether and where the task should be offloaded.For these challenges,the following parts are studied:(1)For the research of task offloading and resource allocation algorithm based on subtask parallelism,a model for jointly optimizing task offloading decision,bandwidth and computation resource allocation is established to minimize the average service time and average service cost of tasks.The data-driven task is decomposed into multiple sub-tasks,which don’t have data dependence on each other and can be processed in parallel.To reduce the time complexity,the problem is solved in two stages.First,an asynchronous deep reinforcement learning algorithm ADQN is proposed to make offloading decisions for tasks,which achieves fast convergence by training local networks in parallel and updating the global network asynchronously.Secondly,the remaining resource allocation problem is decomposed into several independent sub-problems,and the theoretical optimal solution of resource allocation is obtained based on convex optimization theory.The simulation analysis proves the advantage of ADQN.(2)For the research of fast-adaptive task offloading algorithm based on subtask dependencies,an optimization model of task offloading decision is established to minimize the task execution time in different scenarios at the same time.There are multiple task offloading scenarios,each of which is characterized by the features of the vehicular task and the MEC server,such as task topology,data dependencies between subtasks,resource requirements of subtasks,and transmission and computation capacity of the MEC server.To solve this problem,a fast adaptive task offloading algorithm SMRL based on sequence-tosequence and meta reinforcement learning is proposed.The algorithm uses a bidirectional recurrent neural network to iteratively encode sequential inputs to explore data dependencies among subtasks,and utilizes an attention mechanism to evaluate weights on different input parts to make more appropriate offloading decisions for subtasks.In addition,the algorithm also uses the meta learning framework to train a meta offloading strategy offline,which can quickly adapt to new task offloading scenarios within a very short number of training steps.The simulation proves the rapid adaptability of SMRL.