| The Internet has fundamentally changed the way of modern communication. Current trends indicate that high-capacity demands are not going to be saturated anytime soon. From Shannon's theory, we know that information capacity is a logarithmic function of signal-to-noise ratio (SNR), but a linear function of the number of dimensions. Ideally, we can increase the capacity by increasing the launch power, however, due to the nonlinear characteristics of silica optical fibers that imposes a constraint on the maximum achievable optical-signal-to-noise ratio (OSNR). So there exists a nonlinear capacity limit on the standard single mode fiber (SSMF). In order to satisfy never ending capacity demands, there are several attempts to employ additional degrees of freedom in transmission system, such as few-mode fibers (FMFs), which can dramatically improve the spectral efficiency. On the other hand, for the given physical links and network equipment, an effective solution to relax the OSNR requirement is based on forward error correction (FEC), as the response to the demands of high speed reliable transmission.;In this dissertation, we first discuss the model of FMF with nonlinear effects considered. Secondly, we simulate the FMF based OFDM system with various compensation and modulation schemes. Thirdly, we propose tandem-turbo-product nonbinary byte-interleaved coded modulation (BICM) for next-generation high-speed optical transmission systems. Fourthly, we study the Q factor and mutual information as threshold in BICM scheme. Lastly, an experimental study of the limits of nonlinearity compensation with digital signal processing has been conducted. |
