With the penetration of the high-bandwidth networking service including interactive game, high-definition (HD) video on demand (VOD), video conference and tele-education, there exists a severe requirement on the transmission capacity of current optical fiber transmission system. Digital coherent optical fiber transmission, taking the advantage of coherent detection and digital signal processing (DSP), becomes an inevitable choice for the next high speed high capacity optical fiber network. The use of advanced modulation format and mutli-dimension multiplexing technique is enabled by digital coherent optical fiber transmission. Therefore both spectrum eficiency (SE) and capacity of single fiber transmission can be greatly enhanced. Meanwhile, various transmission impairmenbts including chromatic dispersion (CD), polarization mode dispersion (PMD), and fiber nolinearity can be successfully either compensated or mitigated, using DSP-based equalization. In this thesis, we carry out the investigations of several key techniques for the high speed high capacity coherent optical fiber transmission, including advanced modulation format, multi-dimension multiplexing technique and the digital equalization technique. The main work and inovative contributions of this thesis are summarized as follows:(1) We propose a novel Nyquist-wavelength division multiplexing (Nyquist-WDM) transmission technique, using the constant modulus algorithm (CMA) based on multi-input multi-output (MIMO) equalization. Compared to the traditional MIMO equalization using least mean square (LMS) algorithm for the Nyquist-WDM transmission signal, the proposed equalization method is robust against the phase noise of both the optical source at the transmitter side and the local oscillator at the receiver side. Through simulation, the tolerant range of linewidth of the optical comb is20MHz for the Nyquist-WDM signal using CMA-MIMO equalization. Meanwhile, the proposed MIMO equalization can effectively eliminate the inter-channel-interference (ICI) from adjacent channels. Therefore, for the3×224Gbps DP-QPSK Nyquist-WDM signal, by using the CMA-MIMO equalization, a spectrum efficiency of4.48bit/s/Hz can be achieved without sacraficing transmission performance. Furthermore,16QAM signal suffers from more severe ICI and inter-symbol-interference (ISI) than that of QPSK signal. We further propose a LMS-MIMO fee dBack equalization for the16QAM Nyquist-WDM signal. A3×160Gbps dual-carrier DP-16QAM Nyquist-WDM signal can be experimentally transmitted over640km single mode fiber (SMF) link, achieving a SE of6.4bit/s/Hz.(2) We investigate the Nyquist-WDM transmission technique based on maximum likelihood sequence detection (MLSD). Through investigation of Nyquist-WDM signal with non-return-to-zero (NRZ)ã€return-to-zero (RZ) and carrier-supressed return-to-zero (CSRZ) pulse, we verify that by introducing a duobinary digital filter in the choherent reciever before MLSD, the ICI in the Nyquist-WDM transmission can be effectively suppressed and the computational complexity of MLSD can be reduced. Simulation results show that for back-to-back (B2B) transmission, irregardless of pulse modulation formats, similar transmission performance can be obtained for the Nyquist-WDM signal. However, since RZ and CSRZ pulse modulated Nyquist-WDM signal have stronger resistance against fiber nonlinearity, they outperform the NRZ pulse modulated Nyquist-WDM signal with higher Q factors after a long distance SMF transmission.(3) Based on a novel few-mode fiber (FMF) with high mode differential group delay (MDGD), we then propose a novel cascaded MIMO equalization method to suppress the mode crosstalk from the mode division multiplexer/demultiplexer in the mode division multiplexing system. Using the fabricated few mode fiber, we carry out the simulation for an80Gbps two-mode division multiplexing system. Compared to the traditional MIMO equalization, the Q factor can be improved with1.7dB using our proposed cascaded MIMO equalization. The cascaded MIMO equalization method together with the novel FMF can effectively mitigate the crosstalk of mode division multiplexing (MDM) transmission.(4) We put forward a fast and blind chromatic dispersion (CD) estimation method. By superimposing the digital spectrum of the received signal after coherent detection, the accumulated CD after an ultra-long fiber link can be quickly and accuratedly estimated. Through simulation, we show that the proposed CD estimation method operates well for not only the DP-QPSK and DP-16QAM signal with RZ, CSRZ and NRZ pulse shapes but also Nyquist-WDM signals with squared-raised-cosine filtering. Meanwhile, we evaluate the performance of our proposed CD estimation method, based on experimental setup of 7×112Gbps DP-QPSK. WDM signal loop transmission. Only4096samples are experimentally required to estimate the CD within a range of0-105ps/nm. In particular, the remaining residual CD due to the CD estimation error, can be completely compensated by a low-complexity CMA-MIMO based polarization division demultiplexing at the digital coherent reciever. |