Research On Machine Learning Based Nonlinear Compensation Algorithms In Coherent Optical Communication Systems

With the development of cloud computing,big data,artificial intelligence and other technologies,and with the emergence of diversified businesses such as internet commerce,live streaming and ultra-clear video,the network traffic is in the continuous explosive growth at present.Therefore,the development of optical fiber communication system with ultra-large capacity,ultra-high symbol rate and ultra-long distance has become an urgent demand in the field of communication.The coherent optical communication system has the advantages of high receiver sensitivity,large capacity,and the application of multiple high-order modulation formats.And by combined with digital signal processing technology,linear damage such as dispersion in the link can be equalized in the electrical domain,which greatly improve the system performance.However,the performance of optical fiber transmission system is still subject to the Kerr effect,and the nonlinear effects are the ultimate factor limiting the system capacity.Especially for high-order modulation formats,the Euclidean distance becomes smaller,and stronger launch power is required,which resulting in more severe nonlinear distortion of the signal.Therefore,nonlinear equalization is of great significance in the research of realizing high-speed and large-capacity optical fiber transmission systems.Machine learning is regarded as a new direction to solve complex problems in the field of optical communication,and its powerful learning ability can achieve excellent nonlinear compensation effects.In addition,its self-adaptation and adjustment ability can realize flexible real-time compensation,which is the most prominent advantage of machine learning over traditional compensation algorithms based on prior information.Therefore,machine learning techniques is applyed for nonlinear equalization.The main research work of this thesis is as follows:1.Research on nonlinear equalization algorithm based on rough set K-nearest neighbor.The existing problems of equalization algorithm based on K-nearest neighbor are studied,and a novel algorithm based on rough set K-nearest neighbor is proposed.By introducing the concepts of upper and lower approximate space from rough set,the algorithm characterizes the constellation points distribution in training set.And by dividing the overall space,most constellation points can be directly classified,and the calculation range of other constellation points is reduced.The simulation platform of 28 GBaud DP-16QAM and DP-32QAM coherent transmission system are established for verification.Simulation results show that the algorithm has the same equalization effect as the traditional K-nearest neighbor algorithm,and the calculation amount is reduced to about 0.4%to 11.7%,which achieves a significant reduction in computational complexity without affecting the classification accuracy.2.Research on nonlinear equalization algorithm based on recurrent neural network.A nonlinear equalization algorithm based on recurrent neural network is proposed,which utilizes the characteristics of recurrent neural network in processing sequence data and having memory function.This algorithm fully takes the correlation between symbols in transmission in to account.By sending the nonlinear distorted signal and the original transmitting signal into network,the nonlinear damage characteristics can be extracted and the complex mapping relationship between input and output can be found.Finally,a nonlinear equalization model is established to compensate for other distorted signals.Simulation results of 224 Gbps DP-16QAM coherent optical communication system show that the algorithm can effectively equalize nonlinearity in the fiber.And the transmission distance is increased by 320 km at 7%forward error correction threshold compared to the equalization algorithm based on traditional artificial neural network.In addition,this nonlinear equalization model has faster convergence,which is of great significance in practical systems to adapt the changes.In summary,this thesis deeply studies the nonlinear compensation algorithm based on machine learning.An improved K nearest neighbor nonlinear equalization algorithm and a nonlinear equalization algorithm based on recurrent neural network are proposed.These algorithms effectively compensate for the nonlinear phase noise in the fiber,and realize the improvement of system performance and reduction of computational complexity.The research results of this thesis are of great significance for the nonlinear equalization in coherent optical communication systems.