Optimal Design of Dispersion Filter for Time-Domain Implementation of Split-Step Method in Optical Fiber Communication

The nonlinear Schrödinger equation can be solved by split-step methods, where in each step, linear dispersion and nonlinear effects are treated separately in a sequential manner. This thesis investigates the optimal design of an finite-impulse-response (FIR) filter as the time-domain implementation for the linear part. The objective is to minimize the integral of the squared error between the frequency response of the FIR filters and the desired dispersion characteristics over the band of interest. This least square (LS) problem is solved in two approaches: the normal equation approach gives an explicit solution and its Toeplitz structure enables fast computation; the singular value decomposition (SVD) approach provides geometrical, physical and numerical insights based on the theory of discrete prolate spheroidal sequence (DPSS).;We verify the feasibility of the proposed filters in two categories of applications. Firstly, they can be used in the time-domain simulation of pulse propagation in optical fiber. For single channel and WDM channels, the proposed filters generate similar outputs as previous time-domain and frequency-domain methods, even after propagating thousands of kilometers. The proposed designs, together with overlap-add and overlap-save, can reduce the overall computational complexity significantly. Secondly, the proposed filters can also be applied in time-domain digital backpropagation algorithms for fiber impairment compensation. Numerical simulations of a polarization division multiplexed quadrature phase-shift keying (PDM-QPSK) transmission system illustrate that the split-step methods based on the proposed filters are able to effectively mitigate the signal distortions caused by both dispersion and nonlinearities.;A major concern is that as revealed by the theory of DPSS, this problem could be ill-conditioned, and henceforth its solution would be sensitive to small perturbations. Besides, the frequency response might exhibit singular behaviors such as overshoots. Two approaches are proposed to mitigate these shortcomings: the unconstrained LS approach adds a regularization term to the objective function, whereas the constrained approach also imposes a maximum magnitude constraint on the frequency response. The latter approach is formulated into a standard quadratically constrained quadratic programming (QCQP) problem that can be readily solved using state-of-the-art interior-point methods. Compared with previously designed FIR filters, these filters are easier to extract and the QCQP-based filter saves the filter length by at least 1/3. There is a complexity trade-off between these two filters: the unconstrained regularized LS filter is much easier to extract with the help of the modified Levinson-Durbin algorithm; the QCQP-based filter is shorter in length and saves computational complexity of the linear convolutions. The choice depends on whether the filter needs to be regenerated frequently or not. In addition, the required filter lengths for these filters are approximately linear functions of several parameters, which simplifies the task of choosing step size.