Research On Channel Access Technology Of MAC Layer Based On Multi-agent Q-Learning Algorithm For Vehicular Communication

As an important technology to support the applications of ITS(Intelligent Transportation Systems),VANETs rely on the real-time and reliable transmission of safety information to provide solutions to the safety related problems.It's difficult to ensure that the safety information can be transmitted with low delay and high reception rate.Because the message collision occurs easily using the traditional IEEE 802.11 p protocol due to the poor scalability and the fact that the topology of VANETs changes frequently and the nodes move at high speed in the scenario where the vehicle density is high.The efficient MAC protocol is designed in this thesis through building a novel model and improving the traditional channel accessing method focusing on these problems.Firstly,the Q-learning model is established for vehicle nodes through bringing Q-learning algorithm into the process of accessing the wireless channel to transmit data,and then the Q-learning based algorithm of adjusting the size of CW(Contention Window)dynamically is proposed.Moreover,the performance of the proposed algorithm is analyzed and simulated in terms of packets reception rate,transmission delay and collision rate.Secondly,the Q-learning process of single node is of large search space and low efficiency as it perceives only the partial network environment in the absence of learning with other vehicle nodes interactively.Therefore the multi-agent Q-learning based algorithm of adjusting the size of CW dynamically is proposed after establishing the model of Q-learning based multi-agent system for VANETs focusing on these problems,thus the fairness of accessing the wireless channel for vehicle nodes is improved and the proposed algorithm is scalable to various extent of network load.Lastly,Q-learning result of agents converges to correlated equilibrium.The eCEQ(correlated equilibrium Q-learning)algorithm is used to maximize the number of accessing the wireless channel to transmit data successfully for each vehicle node in VANETs.In addition,the convergence of the proposed algorithm is proved to be right through simulation.