Deep Reinforcement Learning For Dynamic Subcarrier Allocation Algorithm In OFDMA-PON

In recent years,the continuous development of science and technology has brought about a sharp increase in the propotion of mobile services and network users,It has imposed stricter requirements on the service quality of traditional network architectures,such as network transmission delays.Due to the advantages of large capacity,efficient and flexible multiple address access,high spectrum efficiency,dynamic bandwidth allocation,etc.Orthogonal frequency division multiplexing access passive optical network(OFDMA-PON)has become one of the most potential choices for the next generation optical access network.In OFDMA-PON,it allows different optical network units(ONUs)to share subcarrier resources to support network resource management and effective bandwidth allocation.In uplink transmission,multiple ONUs can share orthogonal low bit rate subcarriers(SC)in different time slots(TS)in the entire transmission cycle to transmit data.In this thesis,we study the dynamic subcarrier allocation(DSA)strategy based on deep reinforcement learning(DRL)in OFDMA-PON.We comprehensively considered the application prospect of DRL in resource management and the problem of subcarrier resource management in OFDMA-PON.We organically integrated the above two aspects in the DRL-based DSA strategy in OFDMA-PON.First of all,this thesis uses this strategy to allocate time slots and subcarriers jointly in a dynamic and flexible manner.The DRL agent learns to reserve some subcarrier resources for smaller ONU requests in iterative training to reduce the average ONU request waiting time.Next,this thesis considers the choose of modulation format in ONU,the DRL agent learn to dynamically schedule the subcarrier resources in the subcarrier pool according to the bandwidth requirement range of the ONU request.The agent makes the ONU requests choose the lower modulation format under the ensured quality of delayed service,then we can reduce the transmission power of ONU in OFDMA-PON to achieve the purpose of energy saving.In this thesis,the two indicators of ONU transmission power and average delay of ONU request are used to simulate and analyze the DRL-based DSA strategy.At the same time,the normalized total reward indicator is introduced to optimize the above performance indicators comprehensively.At first,we compares the DRL-based DSA algorithm using with the traditional two-dimensional DSA algorithm in OFDMA-PON.Through numerical simulation,the delay performance of ONUs in two-dimensional resource scheduling is first analyzed and demonstrated.Then the choice of modulation format is considered,the delay performance and transmission power of ONUs in three-dimensional resource scheduling is analyzed and demonstrated.The simulation results show that the DSA algorithm using DRL for the first time achieves better service delay and can save more transmission power.In addition,this thesis shows how the strategy can adapt to different conditions from the following aspects: 1)the change of ONUs bandwidth request range,2)the change of ONUs request load,and 3)the change in the weight of different indicators.This thesis gives a more flexible scheduling scheme,delay performance analysis and energy-saving strategy for the research of DRL-based DSA strategy in OFDMAPON,and has achieved good results.It also gives corresponding suggestions for the further use of DRL to optimize the bandwidth allocation algorithm.The strategy has important theoretical guiding significance and application value.