Research On Mobile Opportunistic Network Routing Protocol Based On Reinforcement Learning

Mobile opportunistic networks adopt the "storage-carry-forward" communication mode,which can solve the communication problems in poor network conditions such as intermittent connection,frequent communication link disconnection,lack of infrastructure,etc.Mobile opportunistic networks have application prospects in urban vehicle communication networks,underwater sensor networks,and animal field activity tracking.How the mobile opportunistic networks realize data transmission in complex network scenarios,routing protocols play a vital role.Existing routing protocols mainly rely on node attributes such as contact information and geographic location information to improve network transmission performance,but lack of awareness of the transmission performance of data packets in the network,which easily results in unreasonable forwarding node selection and data packet management and causes data congestion in some nodes.To solve the above problems,the main work of this thesis is as follows.(1)A Forwarding Utility Learning model based on Double Update strategy(FULDU)is proposed.By interacting with the network environment,FULDU obtains local feedback information for node forwarding utility value learning.The contact interval between nodes is used to calculate the node contact probability,and the contact freshness coefficient is defined for the dynamic adjustment of the contact probability.The contact probability is introduced into learning process to adapt to the opportunistic contact characteristics of the mobile opportunistic networks.According to routing requirements,the immediate return value and dynamic discount factor of the learning process are defined and the meaning of these parameters corresponding to the mobile opportunistic networks is analyzed.A dual update strategy of packet forwarding update and node contact update is proposed to update forwarding utility value to improve the learning speed of routing information in the network.Finally,through theoretical analysis,the convergence of the learning process of FULDU model is verified.(2)A node congestion awareness model is designed,and a Probabilistic Routing protocol with Congestion Awareness based on Q-Learning(QLPR-CA)is proposed based on the forwarding utility learning model FULDU.In the node congestion awareness model,the node congestion coefficient and node area congestion coefficient are defined.In the process of data packet forwarding,forwarding strategy based on utility Q value and forwarding strategy based on contact probability are proposed,and the node area congestion coefficient is used for the allocation of copies.The node area congestion coefficient and node contact probability are appliedd to initialize the number of packet copies.In the forwarding strategy based on contact probability,the node contact probability,packet survival time and the number of packet copies are used for the management of the packet forwarding queue.In the forwarding strategy based on utility Q value,the forwarding utility Q value,packet survival time and the number of packet copies are used for the management of the contact probability queue.When the cache is congested,the node contact freshness coefficient,forwarding utility Q value,packet survival time and the number of packet copies are used to manage the overflow delete queue.Finally,the QLPR-CA protocol is implemented in simulation platform ONE and compared with Epidemic,Prophet,Sa W,EBRR,and EURR.Three evaluation indicators of the delivery rate,forwarding cost and average delivery delay of the QLPR-CA protocol and the comparison protocols are analyzed in the simulated data and the real data set Infocom2006.The simulation results show that the overall network performance of the QLPR-CA protocol is more adaptable than other routing protocols in the process of network scenario parameters change,and the network transmission performance is better than other protocols.