Research On Multi-domain Resource Management And Optimization Of Internet Of Vehicles Based On Deep Reinforcement Learning

The flourishing growth of emerging technologies of 5G mobile communication and the phenomenon that vehicles are becoming more and more popular and intelligent make the explosive increase of new and demanding applications generated in the Internet of Vehicle,which brings huge challenges to the vehicles.Because the vehicles do not have enough resources to deal with such applications.Applying Mobile Edge Computing(MEC)technology to the Internet of Vehicles is capable of relieving the stress on the network,of which satisfies the requirements of high-performance computing applications effectively to a certain extent and scope.However,the computational and storage resources of the MEC server are limited.Therefore,it is of great importance to select the appropriate service nodes to compute the vehicle tasks and to allocate multi-domain resources including computational resources,storage resources and communication resources jointly to meet the demands of low-latency and high-speed tasks in the C-V2X(Cellular-Vehicle-to-Everything)scenario.In addition,the subtask scheduling sequence in the data transmission through single antenna,and the order of subtasks processed by different service nodes will make big difference on the task total completion time when offloading multiple subtasks produced by the vehicle to the service nodes.Consequently,this thesis studies the V2X task scheduling and offloading problem,and also studies the efficient optimization and allocation scheme of multi-domain resources in the MEC-based vehicular networking architecture.Firstly,in order to solve the problem of offloading multiple independent subtasks of a single vehicle,this thesis proposes a task scheduling and offloading scheme based on DQN(Deep Q-Network)for the purpose of minimizing the task total completion time.And the proposed scheme can find the best subtask offloading sequence and the suitable service nodes under the guarantee of V2V(Vehicle-to-Vehicle)communication reliability.Besides,this thesis proposes the indicator which called task time overlap rate to analyze the optimization performance of the proposed algorithm,and to explore the realization mechanism of the optimal multi-domain resource allocation scheme as well.It can be seen clearly from the simulation results that the proposed scheme is able to reduce the total task completion delay effectively by dynamically adjusting the offloading strategy in comparison with the other methods,so that it can better adapt to the scenarios of different parameter settings.Secondly,to solve the problem of jointly optimizing the computational resources,storage resources and communication resources to maximize the long-term rewards of the system in the multi-vehicle task offloading scenario where resources are scarce,this thesis proposes a fully cooperative algorithm which called Multi-Agent Deep Recurrent Q-Network(MADRQN).Through applying this algorithm,the base station is able to select the appropriate multiplexing channel and allocate the corresponding virtual machine to deal with the vehicle task,meanwhile,it can store the task calculation results through the designed caching strategy which comprehensively takes the task popularity and the average system benefit of storage into consideration.In addition,considering the task importance and the fairness that the vehicle task can be handled,this thesis designs the vehicle selection principle and task budget adjustment mechanism.The simulation results show that compared with SDRQN(Single-agent Deep Recurrent Q-Network)and random methods,the proposed algorithm has superior system performance,which can obtain 8.1%and 17.2%increase in system revenue,respectively.Moreover,the proposed algorithm can also make an estimate of the optimal number of tasks that are supposed to be cached in the MEC server.